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Abstract 

The  purpose  of  this  research  was  to  evaluate  the 
applicability  of  Bayesian  statistical  methods  to  the  problem 
of  determining  cruise  missile  component  reliability.  There 
were  three  objectives:  1)  to  develop  models  incorporating 
Bayesian  reliability  concepts  that  can  be  used  to  predict 
component  reliability  based  on  data  available  in  a  program 
transitioning  from  development  to  production;  2)  to 
determine  the  model's  validity  in  comparison  with  classical 
statistical  models;  and  3)  to  assess  the  accuracy  of  both 
approaches  against  actual  cruise  missile  flight  test 
history. 

A  total  of  six  models  were  developed  for  the  failure 
rate  of  the  Tomahawk  Cruise  Missile  Guidance  Set  using  both 
exponential  and  binomial  distributions.  The  flight  test 
data  seemed  to  belong  to  another  failure  distribution,  and 
was  not  useful  as  a  measure  of  performance  as  had  been 
proposed. 

The  Bayesian  Expert  Information  Model  provided 
reasonable  point  estimates  of  the  failure  rate  and  markedly 
shorter  90%  confidence  intervals.  Tn  general,  the  Bayesian 
models  had  confidence  intervals  that  were  shorter  than  the 
classical  statistical  inference  models,  allowing  a  more 
accurate  decision-making  process. 

vii 
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Future  effort  in  this  field  could  be  directed  toward 
applying  these  models  to  other  weapon  systems  or  components. 
Other  applications  could  include  using  the  Bayesian  approach 
in  the  Aeronautical  Systems  Division  Avionics  Integrity 
Program  (AVIP)  to  increase  the  Failure  Free  Operating  Period 
of  components,  or  in  the  assessment  of  the  reliability  of 
foreign  technology  based  on  partial  information. 


viii 


APPLICATION  OF  BAYESIAN  RELIABILITY  CONCEPTS  TO  CRUISE 


MISSILE  ELECTRONIC  COMPONENTS 

I.  Introduction 


General  Issue 

The  reliability  of  electronic  components  in  United 
States  Air  Force  weapon  systems  greatly  contributes  to  the 
overall  weapon  system  mission  reliability.  Before  a 
decision  to  proceed  with  weapon  system  production  can  be 
made,  the  Air  Force  has  an  obligation  to  verify  these 
reliability  values  or  demonstrate  that  they  can  reasonably 
be  expected  to  be  attained  during  the  early  production 
testing  of  hardware.  The  type  of  statistical  analysis  used 
to  establish  the  confidence  in  reliability  predictions  can 
influence  the  amount  of  testing  necessary.  The  accepted  Air 
Force  approach  to  date  uses  "classical"  statistical 
inference  techniques  that  require  large  data  samples  to 
generate  reliability  predictions  with  confidence  intervals 
constrained  enough  to  be  useful  for  management  decisions. 

In  order  to  make  a  meaningful  reliability  prediction 
(one  with  a  narrow  band  of  high  confidence)  a  large  amount 
of  test  data  must  be  accumulated.  This  usually  occurs  at  a 
time  when  both  funding  and  test  assets  are  in  critically 
short  supply  and  the  time  required  to  accomplish  the  testing 
can  threaten  program  production  authorization.  Testing  is 
generally  limited;  unidentified  component  deficiencies,  even 
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if  later  detected,  may  impact  operational  readiness  and 
mission  reliability  or  necessitate  costly  retrofits. 

Problem  Statement 

Can  a  reliability  model  based  on  Bayes'  Theorem  be 
developed  that  better  predicts  component  reliability  using 
minimal  test  data  and  provides  the  same  or  better  levels  of 
confidence  than  classical  statistical  inference  methods? 

Research  Objectives 

The  objectives  of  this  research  are:  1)  to  develop 
models  incorporating  Bayesian  reliability  concepts  that  can 
be  used  to  predict  component  reliability  based  on  data 
available  in  a  program  transitioning  from  development  to 
production;  2)  to  determine  the  model's  validity  in 
comparison  with  classical  statistical  models;  and  3)  to 
assess  the  accuracy  of  both  approaches  against  actual  cruise 
missile  flight  test  history. 

Scope 

This  research  will  be  dealing  with  cruise  missile 
electronic  components.  Cruise  missiles  are  unique  hybrids 
of  missiles  and  aircraft  in  that  they  perform  most  of  the 
functions  common  to  manned  aircraft  without  the 
characteristic  designed  redundancy  that  tends  to  increase 
aircraft  mission  reliability.  Additionally,  because  they 
are  largely  dormant  systems,  there  is  relatively  little 
cruise  missile  operational  data  for  use  in  evaluating  both 
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system  and  component  reliability.  There  are  many  different 
approaches  that  incorporate  elements  of  Bayesian  philosophy 
in  statistical  decision  theory.  The  next  section  will 
identify  and  discuss  some  of  these  and  contrast  them  with 
the  more  prevalent  classical  inference  view. 
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II.  literature  Review 


Introduction 

This  literature  review  will  consider  current  thinking 
in  the  use  of  Bayesian  concepts  for  making  reliability 
predictions  of  electronic  components. 

Justification 

The  defense  posture  of  the  United  States  is  based  on 
countering  a  numerically  superior  enemy  with  the  combat 
capability  of  qualitatively  superior  weapon  systems. 

To  win  under  these  circumstances,  systems  must  perform 
not  just  once  but  must  sustain  operational  performance 
over  time.  (7:1) 

This  phrase  defines  reliability  as  it  is  used  in  this 
research  and  reflects  a  renewed  Air  Force  commitment  to 
ensuring  reliability  in  weapon  systems.  Concurrent  with  the 
rising  interest  in  reliability  is  an  increase  in  spending  on 
research,  development,  testing  and  evaluation  of  new  weapon 
systems  that  consumed  about  one-sixth  of  the  Department  of 
Defense  budget  in  1983  (38:1).  Any  savings  in  the  testing 
process  that  do  not  adversely  impact  reliability  performance 
are  clearly  desirable. 

Plan  of  Development 

This  review  will  first  introduce  the  Classical 
Statistical  Inference  and  Bayesian  approaches  from  a 
managerial  perspective.  Reliability  concepts  and 
reliability  testing  requirements  will  be  addressed,  followed 
by  some  classical  statistical  concepts.  It  then  will 
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explain  Bayesian  statistical  concepts  and  the  various 
testing  applications  described  in  the  literature,  finally 
discussing  the  proposed  model  for  this  research. 


Analysis  of  the  Literature 

Managerial  Perspective .  Air  Force  managers  are  tasked 
with  the  responsibility  of  making  weapon  system  acquisition 
decisions.  To  provide  themselves  with  a  basis  for  a 
particular  decision,  they  consult  the  accumulated  expertise 
of  engineers  and  statisticians.  Often,  there  is  a  tendency 
on  the  part  of  the  manager  to  rely  too  heavily  on  the 
analyses,  making  statistically  significant  decisions  that 
are  compromised  by  limitations  of  the  statistical  theory 
employed.  Berger  recounts  an  example  from  Pratt  that 
demonstrates  this: 

An  engineer  draws  a  random  sample  of  electron  tubes  and 
measures  the  plate  voltages  under  certain  conditions 
with  a  very  accurate  voltmeter,  accurate  enough  so  that 
measurement  error  is  negligible  compared  with  the 
variability  of  the  tubes.  A  statistician  examines  the 
measurements,  which  look  normally  distributed  and  vary 
from  75  to  99  volts  with  a  mean  of  87  and  a  standard 
deviation  of  4.  He  makes  the  ordinary  normal  analysis, 
giving  a  confidence  interval  for  the  true  mean.  Later 
he  visits  the  engineer' s  laboratory,  and  notices  that 
the  voltmeter  used  reads  only  as  far  as  100,  so  the 
population  appears  to  be  "censored".  This  necessitates 
a  new  analysis  if  the  statistician  is  orthodox. 

However,  the  engineer  says  he  has  another  meter, 
equally  accurate  and  reading  to  1000  volts,  which  he 
would  have  used  if  any  voltage  had  been  over  100.  This 
is  a  relief  to  the  orthodox  statistician,  because  it 
means  the  population  was  effectively  uncensored  after 
all.  But  the  next  day  the  engineer  telephones  and 
says,  "I  just  discovered  my  high-range  voltmeter  was 
not  working  the  day  I  did  the  experiment  you  analyzed 
for  me."  The  statistician  ascertains  that  the  engineer 
would  not  have  held  up  the  experiment  until  the  meter 
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was  fixed,  and  informs  him  that  a  new  analysis  will  be 
required.  The  engineer  is  astounded.  He  says,  "But 
the  experiment  turned  out  just  the  same  ...  I  learned 
exactly  what  I  would  have  learned  if  the  high-range 
meter  had  been  available.  (3:24-5) 

The  difference  between  the  engineer  and  the 

statistician  in  the  above  example  is  one  of  philosophy.  The 

statistician  is  subscribing  to  the  classical  statistical 

inference  approach,  while  the  engineer  is  intuitively 

describing  the  Likelihood  Principle,  which  is  the  basis  for 

the  Bayesian  approach.  This  principle  will  be  explained  in 

greater  detail  later  in  this  review,  but  it  in  essence  is 

stating  that  only  actually  observed  samples  have  a  bearing 

on  the  determination  of  a  population  characteristic.  The 

key  point  lies  in  recognizing  the  utility  of  statistical 

theory  as  a  tool  for  decision-making  rather  than  as  an  end 

to  the  means. 

Approach  Contrasts .  The  fundamental  difference  between 
the  two  approaches  can  be  seen  in  Figure  1  and  Figure  2. 
Here,  sampling  theory  refers  to  classical  statistical 
inference  theory. 


Assumptive 

Sampling  Model  < - 

i 

A 

1 

V 

» 

1 

Sample  Data  - >  Inductive 

- >  Statistical 

Reasoning 

Inferences 

Figure  1.  Inferences  Based  on  Sampling  Theory  (22:166-8) 
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Assumptive  < - 

Sampling  Model 


Sample  - > 

Data 

Assumptive 

Prior 

Model - > 


— >  Bayes'  — >  Deductive  — >  Statistical 
Theorem  Reasoning  Inference 


Figure  2.  Bayesian  Inference  (22:166-8) 


These  approaches  will  be  described  in  more  detail 
later,  however,  it  can  be  seen  that  the  Bayesian  reasoning 
process  is  deductive  as  opposed  to  the  classical  inductive 
process.  Martz  also  summarizes  other  characteristics  of  the 
two  approaches  in  Table  1 . 

Another  criticism  of  the  classical  approach  is  its  use 
of  what  Berger  calls  "initial  precision"  as  a  measurement  of 
accuracy.  Before  an  experiment  is  run,  the  researcher 
somewhat  arbitrarily  selects  a  decision  rule  that  defines 
the  level  of  precision.  A  common  decision  rule  has  an 
initial  precision  of  90%  when  an  oi  value  =  0.10  is  chosen. 
This  is  appropriate  for  a  long  series  of  identical  tests, 
such  as  sampling  products  on  a  production  line,  where  the 
unknown  true  characteristic  being  tested  would  fall  within  a 
given  confidence  interval  in  90%  of  the  whole  series.  In  a 
one  time  test,  the  "final  precision"  is  of  greater  interest. 
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Table  1.  Summary  of  Certain  Characteristics 
of  the  Sampling  Theory  and  Bayesian  Methods  of 
Statistical  Inference  (22:169) 


Characteristic 

Sampling  Theory 

Bayesian 

Parameter  of  Interest 

Unknown  Constants 

Random  Variables 

Prior  Distribution 

Does  not  exist 

Explicitly  Assumed 

Sampling  Model 

Assumed 

Assumed 

Posterior  Distribution 

Does  not  exist 

Explicitly  Derived 

Reasoning 

Inductive 

Deductive 

Type  of 

Interval  Estimate 

Confidence 

Interval 

Probability 

Interval 

Past  Experience 

Not  Applicable 

Applicable 

Purpose  of  Sampling 
Experiment 

Supply  data  for 
Making  Inferences 

Confirm  or  Deny 
Expected  Perfor¬ 
mance  from  Past 
Experience 

Quality  of  Inferences 

More  Restrictive 
because  of  Ex¬ 
clusive  Use  of 
Sample  Data 

Depends  on  Ability 
to  Quantitatively 
Relate  Past  Ex¬ 
perience  to 

Sample  Data 

Quantity  of  Bayes'  approach  usually  requires  less 

Sample  Data  because  ic  utilizes  relevant  past  data 

i.e.,  what  is  the  probability  that  the  true  value  is 
contained  in  the  90%  (or  any  other)  confidence  interval 
(3:18-9) . 

Berger  identifies  seven  interrelated  reasons  for 
considering  the  Bayesian  approach. 
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(i)  Prior  information  is  too  important  to  ignore  or 
deal  with  in  an  adhoc  fashion. 

(ii)  According  to  most  "classical"  criteria,  the  class 
of  "optimal"  procedures  corresponds  to  the  class  of 
Bayes  procedures,  so  one  should  select  from  among  this 
class  according  to  prior  information. 

« 

(iii)  The  Bayesian  viewpoint  works  better  than  any 
other  in  revealing  the  common  sense  features  of  a 

*  situation  and  producing  reasonable  procedures. 

(iv)  The  goal  of  statistics  is  to  communicate  evidence 
about  uncertainties,  and  the  correct  language  of 
uncertainty  is  probability.  Only  subjective 
probability  provides  a  broad  enough  framework  to 
encompass  the  types  of  uncertainties  encountered,  and 
Bayes  theorem  tells  how  to  process  information  in  the 
language  of  subjective  probability. 

(v)  Axioms  of  rational  behavior  imply  that  any 
"coherent"  mode  of  behavior  corresponds  to  Bayesian 
behavior  with  respect  to  some  prior  distribution. 

(vi)  The  Likelihood  Principle  seems  irrefutable,  yet 
the  only  general  way  of  implementing  it  seems  to  be 
through  Bayesian  analysis. 

(vii)  Bayesian  posterior  measures  of  accuracy  seem  to 
be  the  only  meaningful  measures  of  accuracy.  (37:2) 

The  mechanics  of  the  classical  statistical  inference 

and  Bayesian  approaches  will  be  addressed  in  greater  detail. 

First,  though,  a  general  overview  of  reliability, 

probability  distributions,  and  testing  will  provide 

background  information  for  later  discussions. 

Reliability.  Reliability  is  the  probability  that  a 

system  or  component  operates  without  failure  through  a 

specified  time  period  (1:105).  A  failure  is  the  partial  or 

w  total  loss  of  function  of  a  unit  in  such  a  way  that  its 

|  intended  purpose  is  impeded  or  stopped.  An  electric  light 

I  bulb  is  an  example  of  a  unit  with  a  well-defined  failure 

I 

! 

i 
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mode  (10:71) .  For  the  purposes  of  this  research,  a 
component's  reliability  will  be  evaluated  in  terms  of  its 
life  and  its  ability  to  complete  a  specified  task. 

A  component's  life  is  defined  as  the  difference  in  time 
between  when  a  unit  begins  to  function  and  the  instant  that 
a  failure  occurs  (10:78).  For  a  sample  of  components,  this 
life  may  also  be  expressed  as  a  failure  rate,  which  is  the 
ratio  of  failures  occurring  in  a  period  of  time  to  the 
number  of  units  that  had  no  failures  up  to  that  instant 
(1:105) . 

The  failure  rate  has  the  general  characteristics  of  a 
"bathtub  curve"  when  plotted  against  time  (see  Figure  3) . 

It  starts  with  a  short  period  of  time  with  a  large 
decreasing  failure  rate  akin  to  sliding  down  the  inside  of  a 
bathtub.  This  corresponds  with  a  high  failure  rate  due  to 
"infant  mortality"  or  production  defects  that  cripple  the 
component's  function.  The  bottom  of  the  bathtub  curve 
refers  to  the  useful  life  of  a  component;  it  has  the 
smallest  failure  rate  and  failures  during  this  period  are 
frequently  categorized  as  random  because  the  failure  rate  is 
constant.  At  the  end  of  the  bathtub  curve,  the  failure  rate 
increases  sharply  corresponding  to  the  increasing  number  of 
failures  due  to  aging  (20:84-6).  Many  component  samples 
experience  both  the  infant  mortality  and  aging  phases; 
techniques  such  as  environmental  stress  screening  have  been 
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Figure  3.  Bathtub  Curve  (20:84-86) 


developed  to  handle  the  former  and  components  can  be  removed 
from  the  inventory  prior  to  the  onset  of  the  latter 
(15:3-5) . 

Probability  Distributions .  The  failure  rate  for  the 
useful  life  of  a  sample  of  components  may  be  described  as  a 
random  variable  with  one  of  several  probability 
distributions:  exponential,  binomial,  normal,  gamma,  and 
Weibull.  The  exponential  distribution  (1:106)  has  a 
constant  failure  rate,  ,  that  is  the  reciprocal  of  the 
Mean-Time-Between-Failure  (MTBF) ,  ,  and  reliability. 


R(t)  =  exp(-t*  ) 


(1) 
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The  binomial  distribution  can  be  used  when  a  dichotomy 
exists  between  the  states  a  random  variable  can  assume 
within  a  population.  For  reliability  purposes,  it  can 
describe  situations  where  components  either  successfully 
complete  or  fail  an  event.  Two  assumptions  are  required:  1) 
each  of  n  sample  items  has  the  same  chance  p  of  being  in  a 
certain  state,  and  2)  the  outcomes  of  the  n  sample  items  are 
statistically  independent  (25:91-2). 

For  a  sample  of  n  units,  the  binomial  probability 
function  (25:92)  can  be  expressed  as 

n ! 

f(y)  =  -  py  (i-p)n~y  (2) 

y!  (n-y) ! 

where  y  is  the  number  of  successes.  In  this  case,  p  is  the 
reliability  of  the  component  under  test.  The  other 
distributions  have  specific  applications  that  will  not  be 
discussed  here  (1:106-7). 

"The  instant  at  which  a  failure  will  occur  cannot  be 
told  in  advance:  this  instant  is  random  (10:145)."  Although 
we  can  describe  the  general  failure  probability 
distributions,  reliability  testing  is  necessary  to  estimate 
specific  "numerical  reliability  characteristics  (10:145)." 

Reliability  Testing.  Testing  is  used  to  verify  the 
validity  of  theoretical  reliability  calculations,  usually  in 
an  environment  as  nearly  representative  of  the  operational 
scenario  as  is  possible.  Ling  and  Arsenault  identify  four 
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underlying  principles  of  reliability  testing,  including  the 
use  of  "statistically  efficient  tests  chosen  to  minimize 
cost  and  time  to  an  accept/reject  decision  (1:336-7)." 

4  MIL-STD-785  also  states  that  reliability  testing  should 

be  "tailored  for  effectiveness  and  efficiency  (maximum 

4. 

return  on  cost  and  schedule  investment)  in  terms  of  the 
management  information  they  provide  (8:A-25)."  The  military 
standard  (8:A-25)  points  out  a  difference  between 
reliability  values  measured  in  early  Reliability  Design 
Growth  Testing  (RDGT) ,  which  are  not  expected  to  correlate 
with  operational  values,  and  those  from  final  RDGT  and 
Reliability  Qualification  Testing  (RQT)  which  must  possess  a 
clear  traceability  to  field  requirements. 

Reliability  Confidence  Testing  (RCT)  is  a  shortened 
form  of  RQT  sometimes  used  when  funding  and  t ime  do  not 
permit  a  complete  RQT  as  described  by  MIL-STD-781.  The 
Cruise  Missile  Guidance  Set  Reliability  Demonstration  will 
be  considered  to  be  this  kind  of  a  test.  The  next  section 
describes  the  hardware  components  and  the  objectives  of  this 
test . 

Cruise  Missile  Guidance  Set  Reliability  Demonscration. 
The  Tomahawk  Cruise  Missile  is  an  airbreathing  turbofan 

*  missile  designed  to  fly  at  high  subsonic  speed  and  low 

altitude,  striking  targets  with  high  accuracy  at  distances 

*• 

up  to  1500  nautical  miles.  There  are  conventional  and 
nuclear  variants  developed  for  the  U.S.  Air  Force  and  Navy 
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designed  to  be  launched  from  ground  vehicles,  ships,  and 
submarines.  This  research  will  consider  only  the  Tactical 
Land  Attack  Missile  (TLAM)  variants. 

The  heart  of  the  TLAM  missile  is  the  Cruise  Missile 
Guidance  Sec  (CMGS) ,  which  consists  of  a  gyro  assembly,  a 
radar  altimeter,  and  an  on  board  microprocessor/computer . 
Prior  to  launch,  the  launch  location  is  loaded  into  the 
system  and  during  flight  the  CMGS  uses  terrain  correlation 
for  navigation  updating.  While  flying  over  land,  the 
downlcoking  radar  computes  terrain  altitude  profiles  that 
are  compared  with  computer  stored  digital  map  data, 
providing  the  basis  for  corrections  to  the  missile 
flightpatn.  The  CMGS  went  through  a  substantial 
configuration  change  midway  through  production,  partially 
due  to  accumulated  producibility  changes  and  minor 
reliability  improvements.  Based  on  a  perceived  reliability 
deficiency  and  lack  of  confidence  in  the  configuration 
change,  the  Secretary  of  the  Air  Force  directed  that 
reliability  demonstration  testing  be  accomplished  on  both 
CMGS  configurations  prior  to  field  retrofit  (21) . 

The  CMGS  Reliabixity  Demonstration  had  two  objectives: 

1)  provide  a  point  estimate  of  the  CMGS  mission  MTBF ,  and 

2)  determine  the  field  environmental  effects  on  mission 
reliability.  The  test  was  divided  into  two  phases 
corresponding  to  the  objectives;  the  primary  difference 
being  the  sequence  and  severity  of  the  thermal  and  vibration 


cycles.  These  cycles  were  organized  in  such  a  fashion  as  to 
simulate  the  environment  experienced  by  the  CMGS  in 
operational  use.  The  sample  to  be  tested  consisted  of  units 
taken  from  the  field  and  verified  to  be  in  working  order 
prior  to  the  test.  Because  of  time  and  funding  limitations, 
a  limited  number  of  sample  units  was  used.  In  the  event  of 
a  failure,  the  failed  unit  was  repaired  and  put  back  into 
tne  test.  Test  data  was  collected  according  to  both  the 
number  of  mission  scenarios  successfully  completed  and  total 
time  in  test  for  each  test  asset.  Specific  details  are 
contained  in  the  test  plan  (31:1.1-3.23). 

Classical  Statistical  Inference 

This  section  will  provide  some  background  material  on 

the  classical  statistical  inference  approach. 

...inferential  statistical  analysis  is  concerned  with 
measuring  the  characteristics  of  only  a  sample  from  the 
population  and  then  making  inferences,  or  estimates, 
about  the  corresponding  value  of  characteristics  in  the 
population  from  which  the  sample  was  drawn.  (13:9-10) 

The  total  number  of  components  in  use  in  all  of  the 

weapon  systems  can  be  defined  to  be  the  population.  From 

this  population,  a  subset,  or  sample,  of  components  can  be 

selected  for  testing  when  it  is  not  practical  to  test  or 

measure  each  population  member.  This  is  certainly  the  case 

when  trying  to  determine  the  reliability  of  one-time  use 

items  like  cruise  missiles.  The  sample  is  selected  at 

random,  i.e.  each  member  of  the  population  has  an  equal 
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chance  of  being  included  in  the  sample,  to  validate 
inferences  that  may  be  made  to  the  population  parameters. 

There  are  two  types  of  inferences  that  may  be  drawn 
from  sample  statistics  to  population  parameters  —  point 
estimates  and  interval  estimation.  A  point  estimate  is  a 
single  value  that  best  approximates  the  true  value  of  the 
population  parameter,  but  does  not  indicate  how  much 
uncertainty  is  present.  An  interval  estimate  can  specify  a 
probability  that  the  sample  statistic  is  within  a  certain 
interval  about  the  true  population  parameter.  A  point 
estimate,  such  as  the  mean  of  a  distribution,  x,  will  differ 
from  the  population  parameter,/*  ,  by  some  unknown  amount  of 
sampling  error,  x  =  ±  £  .  If  the  sample  size  is  large, 

the  sampling  distribution  and  the  distribution  of  sampling 
errors  will  both  be  nearly  normal  by  the  Central  Limit 
Theorem.  This  results  in  an  expression  for  the  confidence 
interval  based  on  the  population  standard  error  , 

(x  -  z*cr)  <  }*.  <  (x  +  z* <T)  (3) 

where  z  is  the  critical  value  associated  with  a  specified 
probability.  In  most  cases,  the  population  standard  error 
is  not  known,  and  must  be  estimated  using  the  sample 
standard  error,  s.  The  interval  estimate  obtained  using  s 
is  called  an  approximate  confidence  interval  (13:135-143) . 
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Bayesian  Statistics 


This  section  will  discuss  Bayes'  Theorem  and  several 
approaches  toward  its  implementation  in  reliability  testing. 
One  model,  in  particular,  will  be  described  that  seems  to  be 
useful  for  this  research. 

Bayes'  Theorem.  The  Bayesian  technique  is  easy  to 

explain,  but  its  ramifications  are  more  complex. 

Suppose  that  there  exists  a  set  of  mutually  exclusive 
and  exhaustive  events  that  are  considered:  it  is  known 
in  advance  that  one,  and  only  one,  of  these  events  will 
actually  occur,  but  there  is  uncertainty  about  which  of 
these  it  will  be.  One  begins  by  assigning  a 
probability  to  each  of  these  events  on  the  basis  of 
whatever  evidence  is  currently  available.  Then  if 
additional  evidence  is  subsequently  obtained,  the 
initial  probabilities  are  then  revised  on  the  basis  of 
this  evidence  by  means  of  Bayes'  Theorem.  The  initial 
probabilities  are  known  as  prior  probabilities  in  that 
they  are  assigned  before  the  acquisition  of  the 
additional  evidence  bearing  on  the  problem.  The 
evidence  on  which  these  probabilities  are  based  is 
therefore  prior  information....  The  probabilities  which 
result  from  the  revision  process  are  known  as  posterior 
probabilities.  (24:1-2) 

Bayes  theorem  (4:10)  can  be  expressed  by  the  following 
equations : 


P(yl*)P(X) 

P(Xly)  =  -  (4) 

p  (y) 


where  P(Aly)  is  the  posterior  distribution  of  the 

population  given  sample  data  y, 

P(A)  is  the  prior  distribution  of  A, 

P(ylA)  is  the  likelihood  function  of  A  given  sample 
data  y,  and 


P(y)  is  a  normalizing  constant  necessary  to  make 
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the  posterior  distribution  integrate  or  sum 
to  one. 


P(y)  = 


P(y!X)P(X)d\  X  continuous 

P(ylX)P(X)  X  discrete 


Likelihood  Principle .  In  equation  (4)  above,  the 

posterior  distribution  is  proportional  to  the  prior 

distribution  multiplied  by  the  likelihood  function.  Here 

the  likelihood  function  is  defined  as  the  probability  of 

observing  the  sample  data  y  given  the  true  population  A- 

The  likelihood  function. . .plays  a  very  important  role 
in  Bayes'  formula.  It  is  the  function  through  which 
the  data  y  modifies  prior  knowledge  of  X;  it  can 
therefore  be  regarded  as  representing  the  information 
about  X  coming  from  the  data.  (4:11) 

Prior  Distributions.  The  continuous  prior  distribution 
is  often  discretized  to  allow  more  tractable  data 
manipulation.  Chay  contends  there  is  no  longer  any  need  to 
approximate  continuous  prior  distributions  through 
discretization  because  current  computer  calculations  have 
made  their  numerical  integration  accessible.  Additionally, 
care  must  be  taken  in  subdividing  the  distribution  into 

'V 

discrete  masses  to  ensure  the  areas  of  large  mass  accurately 
portray  true  values  (5:218) .  Conjugate  distributions  are 
often  used  for  priors  because  the  posterior  distributions 
can  be  obtained  relatively  simply  using  analytical  tools. 
Again  alluding  to  the  availability  of  computational  tools. 
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Chay  makes  the  point  that  alternate  distributions  should  be 
considered  as  priors  (5:216) . 

Martz  and  Waller  identify  characteristics  of  successful 
Bayesian  reliability  analyses  that  include: 

1.  A  detailed  analysis  of  the  prior  that  will  be  used 

v  with  documentation  of  data  sources; 

2.  Use  of  a  preposterior  analysis  to  test  the  prior 
with  hypothetical  data; 

3.  Clear,  definitive  descriptions  of  the  posterior 
distribution,  with  a  sensitivity  analysis  of  the  resulting 
Bayesian  inferences  with  respect  to  the  original  prior 
(22:189) . 

The  preposterior  analysis  is  used  to  assess  the  impact 
that  conflicting,  contradictory,  or  confirming  data  samples 
have  when  applied  to  the  proposed  prior.  It  consists  of  the 
following  steps: 

1<  Using  the  proposed  prior,  consider  two  sets  of 
hypothetical  test  data  that  seem  to  be  likely  and  unlikely. 

2.  Compute  the  resulting  posterior  distributions  using 
Bayes'  theorem  for  each  case. 

3.  Examine  the  posterior  distributions  for 
reasonableness  based  on  the  hypothetical  test  data.  If  the 
results  seem  inconsistent,  adjust  the  prior  and  repeat  the 
preposterior  analysis  (22:187).  This  adjustment  often  is 

|  *  the  target  of  allegations  of  test  data  manipulation.  It  is 

j, 

|  important  to  keep  the  perspective  that  Bayesian  reliability 

1 

! 

I 

i 

i 
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analyses,  like  those  used  in  classical  statistics,  are  tools 
for  decision  making,  rather  than  expressions  of  an  absolute 
truth. 

Bayesian  Reliability  Testing  Approaches .  This  section 
will  address  the  current  approaches  in  implementing  Bayesian 
concepts  in  reliability  testing. 

Classical  testing  was  contrasted  with  the  Bayesian 
approach  by  Wonnacott.  He  pointed  out  the  classical 
dependence  on  the  specified  oC  or  Type  I  error  associated 
with  rejecting  a  true  hypothesis.  Often,  the  level  of 
acceptable  error  is  less  easily  determined  than  the 
hypothesis  value  being  tested  (37:149-157).  McCrory  made  an 
impassioned  plea  for  the  use  of  Bayesian  techniques  to  save 
"reliability  testing  from  an  early  death  by  providing  a 
means  to  measure  and  express  reliability  quantitatively  in  a 
more  timely  and  cost  effective  fashion  (23:20)  . " 

Launer  and  Singpurwalla  used  Bayes'  Theorem  to  propose 
a  model  for  detecting  a  deterioration  in  the  system 
reliability  of  one-shot  missiles.  Additionally,  they  wanted 
to  determine  the  marginal  change  in  reliability  since  the 
last  test  and  to  minimize  the  number  of  samples  used  for  the 
destructive  testing  (19:23-6) .  Another  researcher  working 
with  one-shot  devices  proposed  a  series  of  reliability  test 
plans  which  "...will  be  Bayesian  in  the  sense  that  relevant 
data  from  previous  testing  is  to  be  used  in  attempt  to 
reduce  the  sample  size  required  in  subsequent  testing 
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(38:3)."  He  defined  producer's  and  consumer's  risks  and 
developed  confidence  intervals  for  the  reliability 
predictions  (38:8-14). 

Three  research  studies  have  addressed  the  problem  of 
determining  system  reliabilities  based  on  component  or 
subsystem  data.  Lampkin  has  considered  the  use  of  several 
expansion  techniques  to  maintain  precision  in  the 
calculation  of  exact  Bayesian  intervals  as  the  number  of 
subsystems  in  the  reliability  estimate  increases  (18:313-7). 

Winterbottom  was  also  interested  in  estimating  Bayesian 
intervals  for  system  reliabilities  based  on  component  data. 
He  used  exact  and  approximate  methods  both  in  a  series  of 
structures  (series,  parallel,  quorum,  and  others)  and  in  the 
case  of  mixed  testing  of  components,  where  some  are  tested 
pass/fail  and  some  have  exponentially  distributed  failure 
times  (36:1-43).  Barlow  has  identified  a  couple  of  fast 
algorithms  to  calculate  Bayesian  system  reliabilities  based 
on  individual  component  reliabilities.  He  investigated  the 
effect  of  increasingly  large  numbers  of  components  within 
networks  (2:375-383) . 

A  Rome  Air  Development  Center  report  discusses  the 
Bayesian  approach  to  structuring  reliability  tests.  It 
defines  the  producer's  and  consumer's  risks  and  provides 
guidelines  for  selecting  test  risks.  Tne  report  also 
compares  classical  statistical  tests  with  Bayesian  tests  and 
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concludes  that  the  condition  of  the  prior  probability  data 
will  dictate  which  approach  is  optimum  (6:5-35) . 

Johnson  and  Utts  were  concerned  with  estimated  Bayesian 
means  and  confidence  intervals  of  a  normal  population  using 
contaminated  data.  They  used  improper  priors,  which  are 
also  relatively  non-informative,  and  derived  a  posterior 
that  is  approximated  by  a  student  distribution  with  specific 
degrees  of  freedom,  location  and  dispersion.  The  means  and 
confidence  intervals  were  demonstrated  to  be  very  robust  and 
tolerant  of  outliers  (12:8-16). 

Kaplan  proposed  a  "two  stage"  Bayesian  procedure  that 
could  be  used  to  develop  a  prior  when  there  is  little 
generalized  industry  data  available.  In  his  experiment, 
there  was  a  machine  failure  rate  that  could  be  determined 
from  three  types  of  information.  The  first  type, 
engineering  knowledge  of  the  design  and  construction  of  the 
machine,  was  used  to  prepare  the  first  stage  prior.  This 
prior  was  updated  by  the  second  type  of  information,  past 
test  experience  of  similar  machines,  to  form  the  first  stage 
posterior.  This  posterior  was  then  used  as  the  second  stage 
prior  and  updated  to  form  a  posterior  with  the  third  type  of 
information,  current  machine  test  data.  The  first  prior  was 
the  only  distribution  based  on  subjective  rather  than 
quantitative  inputs  (16:1-3). 

Kaplan  also  explored  the  use  of  Bayesian  probabilistic 
analysis  to  determine  the  failure  rate  of  components  in 
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nuclear  power  plants.  There,  he  also  assumed  a  probability 
distribution  was  provided  by  an  expert  source  for  use  as  the 
prior.  Uniform  and  nonuniform  populations  were  considered 
along  with  constant  and  age  dependent  failure  rates 
(14:5-20) . 

Expert  Information  Model .  A  model  has  been  proposed  by 

Kaplan  for  determining  the  reliability  of  electronic  "black 

boxes".  In  its  original  form,  it  is  a  variation  of  a 

binomial  approach.  This  research  will  also  consider  a 

derivative  using  an  exponential  distribution.  A  summary  of 

the  binomial  approach  will  be  addressed  first. 

Our  real  interest  ultimately,  of  course,  is  the 
reliability  of  the  complete  system.  This  will  become 
known  finally  only  from  experience  (i.e.  tests  and 
operations),  with  the  complete  system  itself.  However, 
such  tests  and  operational  data  are  usually  very 
expensive.  Moreover,  we  often  need  to  predict  the 
reliability  of  the  complete  system  at  early  stages  of 
design  and  production,  before  any  (or  certainly  many) 
full  system  tests  are  done.... It  is  often  the  case 
that,  while  we  have  little  experience  with  the  complete 
system,  we  have  considerably  more  information  about  the 
individual  boxes  composing  the  system.  (15:1) 

The  model  is  first  introduced  as  a  thought  experiment 

in  which  a  large  number  of  identical  boxes,  Nq,  are 

repeatedly  put  through  an  operationally  realistic  duty 

cycle,  or  "mission".  The  number  of  boxes  surviving  a  given 

cycle  i,  N^,  divided  by  Nq  can  be  plotted  against  i  cycles 

bn  semilog  paper  to  obtain  a  graph  that  represents  the  3 

phases  of  a  component's  life  —  infant  mortality,  useful 

life,  and  wearout  (see  Figure  4) . 
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Figure  4 .  Survival  Curve  for  Boxes 

(15:3) 

This  model  assumes  the  useful  life  portion  of  the  graph 
has  a  slope,  \  ,  that  is  "the  basic  'failure  rate'  of  the 
box  measured  in  units  of  failures  per  box  per  cycle  (15:3)." 

N.  -  N. 

=  - -  (i  and  i+1  in  useful  life)  (5) 

N. 

i 

Note  that  this  definition  is  not  the  same  as  was  used 
earlier  in  Figure  3  for  the  exponential  model,  where  K  was 
measured  as  a  function  of  operating  time.  It  also  assumes 
that  the  slope,  and  therefore  X,  is  nearly  constant  over  the 
useful  life  of  the  box. 

Kaplan  (15:7)  establishes  a  discretization  of  the 
distribution  of  lambda  values  from  0  to  1.0  for  use  in  the 
following  Bayesian  expression: 
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Pj  <  A  IE) 


(6) 


PgttyPfElA^) 

P0(Ai)p(E|>k .) 

j  J  J 

where, 

p(A- IE)  =  the  posterior  probability  that  the  true 

value  of  A  is  \j,  conditional  on  evidence  E, 

p  (  .)  =  the  prior  probability  that  the  true  value  A 

^  is  Xj  before  evidence  E  existed,  ° 

E  =  the  evidence  or  information  discovered,  and 

p  <E |  Aj)  =  the  likelihood  that  given  the  true  value  of  A 

is  A^  the  evidence  E  would  be  obtained. 

The  first  model  uses  an  exponential  distribution  as  the 
likelihood  function.  Assuming  is  the  number  of  boxes 
that  remain  unfailed  at  time  t,  and  Nq  is  the  number  present 
at  time  zero,  then  =  Nq  e  ^  and  there  is  a  function 
f(t)  such  that  f(t)  =  Nt/NO  =  e  .  This  is  the  fraction 
of  systems  that  remain  unfailed  at  time  t  (17:11).  The 
probability  of  k  failures  in  time  t  can  be  represented  by 
the  Poisson  distribution  (27:8). 

-At 

(  Xt )  6 

p,  (t)  = -  (7) 

K  k! 

This  yields  the  following  Bayesian  posterior  expression  for 
the  exponential  model  (17:25) : 
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e 


(8) 


PtA-j)  (Xjt)k 


-<Ajt> 


n  k  -  ( X *t) 

?L1p(Aj)  (Xjt)  6 


The  likelihood  function  (15:7)  for  the  binomial  model 
can  be  expressed  for  E  =  {k  failures  in  N  cycles}  as 

N!  <X.)k  (l-*.)N~k 

P  <E |  X .)  - - -1 - 2 -  (9) 

J  k! (N-k) ! 

Kaplan  identifies  some  possible  types  of  evidence  that 
may  provide  information  about  the  reliability  of  boxes  (see 
Table  2).  The  following  sections  describe  Kaplan's  approach 
to  using  each  of  these  evidence  sources.  In  each  case, 
evidence  that  is  considered  less  precise  is  used  to  form  a 
prior  distribution  that  is  then  updated  based  on  the  new 
evidence. 

Type  El  -  this  is  data  that  is  presumed  to  be  most 
representative  of  the  true  box  failure  rate.  The  prior 
probability  distribution  would  be  based  on  evidence  types 
E2...E11  and  updated  using  the  failure  rate  calculated  from 
El  data  (either  binomial  or  exponential)  to  estimate  A 
(15:8)  . 

Type  E2  -  this  is  data  that  was  obtained  from  simulated 
testing.  A  similar  process  is  followed  in  using  a  prior 
distribution  generated  from  evidence  E3...E11,  but  in  this 
case,  the  posterior  distribution  will  be  more  representative 
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Table  2.  Possible  Types  of  Evidence 
Relevant  to  Box  Reliability  (15:2' 


Evidence  Description 

El  Direct  experience  of  performance  of  the  box,  as 

part  of  the  complete  system  in  actual  use. 

E2  Experience  of  performance  of  the  box,  as  part  of 

the  complete  system,  but  not  in  actual  use. 

E3  Experience  with  performance  of  the  box  in  isola¬ 

tion,  outside  the  system,  simulating  actual  use. 

E4  Performance  of  the  box  in  screening  tests. 

E5  Performance  of  the  box  in  other  tests. 

ES  Knowledge  of  the  reliability  of  subcomponents. 

E7  Performance  of  earlier  versions  or  designs. 

E8  Experience  of  the  same  or  similar  boxes  in 

other  operating  environments. 

E9  Calculations,  physics  of  failures,  analyses. 

E10  Expert  judgements,  general  engineering 

knowledge,  failure  mechanisms,  etc. 

Ell  Other. 


of  a  failure  rate  A2  for  the  box  in  a  simulated  environment. 
Accordingly,  the  resulting  posterior  distribution  may  be 
modified  by  engineering  judgement  to  reflect  the  increased 
uncertainty  before  it  is  used  as  a  prior  for  evidence  El 
(15:13) . 

Type  E3  -  this  data  generally  comes  from  testing  such 
as  Production  Reliability  Acceptance  Testing  (PRAT)  and 
RQT/RCT .  PRAT  is  used  to  verify  that  production  processes 
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nave  not  degraded  the  designed  reliability  that  was  verified 
through  RQT/FCT .  The  former  consists  of  a  large  number  of 
boxes  tested  once,  while  the  latter  is  usually  a  small 
number  of  boxes  tested  repeatedly.  In  both  cases,  the  focus 
is  on  the  number  of  cycles  successfully  completed.  As  for 
evidence  E2,  the  additional  uncertainty  due  to  differences 
in  testing  can  be  abrogated  through  modification  of  the 
posterior  before  it  is  used  as  a  prior  (15:14-16) . 

Type  E4  -  this  data  usually  reflects  the  effects  of  an 
environment  that  is  more  severe  than  required  operationally. 
As  described  earlier,  manufacturers  use  Environmental  Stress 
Screening  to  precipitate  failures  that  will  occur  in  the 
infant  mortality  period.  The  process  is  repeated  until  a 
unit  passes  through  the  cycles  without  a  failure.  ESS  can 
then  be  defined  as  consisting  of  a  burn-in  cycle  (which 
should  have  a  significantly  greater  X )  and  a  failure  free 
cycle  (which  should  have  a  A  closer,  but  still  higher  than 
the  operational  Xo  value) .  Rather  than  using  these  to 
update  a  prior  distribution  from  other  evidence,  it  is  more 
useful  to  plot  the  distributions  as  upper  bounds  for 
(15:17-20) . 

Type  E5  -  this  type  of  data  can  be  handled  various 
ways,  depending  on  the  characteristics  of  the  testing.  If 
it  is  a  box  level  acceptance  test  at  ambient  conditions,  it 
may  be  possible  to  establish  the  lower  bounds  of  A  for  use 
in  the  same  fashion  as  evidence  E4.  If  the  test  is  more 
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representative  of  the  operational  environment,  it  can  be 
used  to  update  a  prior  based  on  evidence  E6...E11  and  form 
the  prior  for  evidence  E3  (15:21). 

Type  E6...E11  -  this  data  is  usually  less  precise  than 
earlier  test  data.  However,  the  use  of  every  available 
piece  of  information  is  totally  in  consonance  with  the 
Bayesian  philosophy.  The  specific  application  of  each  type 
of  analysis  or  prior  experience  with  an  earlier  generation 
of  the  box  will  depend  on  the  circumstances.  Suffice  to 
say,  the  resulting  distribution  should  be  fairly  broad  to 
reflect  its  highly  uncertain  nature  (15:22-23) . 

Having  defined  the  problem  area  and  some  of  the  ongoing 
work  in  the  field  of  Bayesian  statistics,  it  is  logical  at 
this  time  to  proceed  to  a  discussion  of  the  methodology 
followed  in  this  research. 


III.  Methodolo 


Overview 

This  chapter  describes  the  methodology  used  to 
accomplish  the  above  stated  research  objectives.  This  brief 
overview  of  the  methodology  will  be  followed  by  descriptions 
of  the  data  which  will  be  analyzed.  The  classical 
statistical  inference  approach  will  be  examined  from  two 
points  of  view:  as  a  binomial  distribution  model,  and  as  a 
constant  failure  rate  exponential  distribution  model.  In 
addition  to  the  binomial  and  exponential  distribution  models 
referenced  above,  the  Bayesian  reliability  analysis  will 
also  consider  an  expert  information  model.  A  discussion  of 
the  types  of  data  available  as  sources  to  a  Bayesian  expert 
information  model  will  be  followed  by  a  description  of  the 
steps  involved  in  developing  the  model  itself.  A  comparison 
of  the  point  estimates  and  confidence  intervals  obtained 
from  all  the  Classical  and  Bayesian  models  will  be  made  with 
respect  to  those  obtained  from  actual  flight  test  history. 

Data  Sources 

The  data  obtained  from  the  CMGS  Reliability 
Demonstration  will  be  used  to  make  statistical  inferences 
about  the  true  failure  rate  and  reliability  of  the  CMGS. 

The  three  phases  of  this  test  will  each  be  examined 
separately  and  finally  combined  into  a  single  data  set  in 
each  of  the  following  six  different  models:  1)  the 
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classical  exponential,  2)  the  classical  binomial  3)  the 
Bayesian  exponential,  4)  the  Bayesian  binomial,  5)  the 
Bayesian  expert  information  exponential,  and  6)  the  Bayesian 
expert  information  binomial  models. 

Flight  Test  Data 

The  flight  test  database  consists  of  data  from  three 
variants  of  the  Tomahawk  Cruise  Missile  which  use 
essentially  common  configurations  of  the  CMGS.  Flight  test 
data  for  the  current  configuration  will  be  used  to  calculate 
a  point  estimate  and  confidence  interval  for  the  true 
failure  rate  and  reliability.  Because  the  data  can  be 
evaluated  both  in  terms  of  flight  time  before  failures  and 
number  of  successful  flights,  both  exponential  and  binomial 
distributions  will  be  used. 

The  flight  times  have  been  accurately  recorded,  but 
those  times  are  not  true  indicators  of  the  CMGS  unit  ages. 
The  CMGS  computer  does  have  an  Elapsed  Time  Indicator  (ETI), 
but  the  clock  is  not  reset  every  time  the  unit  is 
refurbished  and  there  appears  to  be  little  correlation 
between  the  failure  times  and  the  ETI  clock  readings.  Since 
the  exponential  model  is  based  on  a  constant  failure  rate, 
the  true  age  of  the  unit  is  not  required  as  long  as  it  can 
be  determined  to  be  in  an  operational  condition  before  the 
flight  test  occurs.  This  would  be  sufficient  to  establish 
the  baseline  from  which  estimation  of  the  failure  rate  could 
proceed. 
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The  flight  test  data  will  be  evaluated  to  attribute 
each  failure  to  a  specific  component.  Components  will  be 
presumed  to  have  been  functional  in  all  successful  flight 
tests  or  flight  test  failures  due  to  other  components  or 
factors.  Assuming  the  missile  achieved  its  system  level 
reliability  requirement,  the  flight  test  results  will  be 
presumed  to  be  an  accurate  reflection  of  the  component's 
operating  environment  and  mission  reliability. 

Flight  Test  Exponential  Point  Estimate .  In  order  to 
use  an  exponential  distribution,  an  evaluation  of  the  flight 
test  failure  data  must  be  made.  Only  the  failed  CMGS  units 
and  their  associated  failure  times  will  be  compared  with  a 
hypothetical  random  distribution  derived  from  the  sample 
failure  rate,  R/T,  as  in  (12) .  The  sample  failure  rate 
includes  test  time  from  units  under  test  that  did  not  fail. 

A  Kolmogorov-Smirnov  Goodness  of  Fit  test  will  be  run  to 
determine  whether  the  failed  items  follow  an  exponential 
distribution.  If  there  is  not  sufficient  cause  to  reject 
the  assumption  that  it  is  an  exponential  distribution,  then 
the  sample  failure  rate  will  be  used  as  the  maximum 
likelihood  estimator  for 

Flight  Test  Exponential  Confidence  Interval .  The  same 
confidence  interval  equation  will  be  used  to  estimate  the 
interval  for  90%  confidence  as  in  equation  (13)  above.  The 
confidence  limits  for  the  exponential  distribution  (22:122) 
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may  be  approximated  using  the  X  (n)  distribution  where 

2  2 
X  (n)  is  the  100(1-*)  percentile  of  the  X  (n)  distribution. 

The  100(1-*)%  TCI  for  is 

X2 (2R)  X2  (2R+2) 

— - - ;  -  (23) 

2t  2t 

Models 

The  first  two  classical  models,  (1  &  2) ,  will  use  only 
the  CMGS  Reliability  Demonstration  data  to  derive  point 
estimates  and  confidence  intervals  for  the  true  failure  rate 
of  the  population.  The  Bayesian  exponential  and  binomial 
models,  (3  &  4),  will  use  early  flight  test  results  as  the 
basis  for  the  priors.  They  will  then  be  updated  with  the 
CMGS  Reliability  Demonstration  data  to  form  the  posterior 
distributions,  from  which  point  estimates  and  confidence 
intervals  will  be  obtained.  The  Bayesian  expert  information 
models,  (5  &  6) ,  will  consider  a  variety  of  data  sources  as 
described  by  the  Kaplan  model,  (see  Table  2) ,  to  establish  a 
series  of  priors  culminating  in  a  final  prior.  The  initial 
prior  will  be  established  based  on  information  evaluated  by 
an  expert  source.  As  in  models  (3  &  4),  the  prior 
distribution  will  be  updated  with  the  results  of  the  CMGS 
Reliability  Demonstration,  using  both  exponential  and 
binomial  distributions.  Point  estimates  and  confidence 
intervals  for  the  failure  rate  will  be  determined  form  the 
resulting  posterior  distributions.  The  mechanics  of 
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calculating  the  classical  and  Bayesian  approaches  will  be  as 
described  in  the  following  sections. 

Classical  Models 

A  reliability  prediction  will  be  made  based  on  a  point 
estimate  of  the  sample  data,  using  a  maximum  likelihood 
estimator.  The  uncertainty  pertaining  to  the  prediction 
will  be  described  in  terms  of  a  two-sided  confidence 
interval.  The  predictions  will  be  accomplished  using  two 
assumed  probability  distributions.  The  first  will  be  an 
exponential  distribution  with  a  constant  failure  rate  and 
the  second  will  be  a  binomial  distribution.  The  total  time 
in  the  test  and  the  corresponding  failures  will  be  used  for 
the  exponential  distribution,  while  the  successful 
completion  of  each  individual  portion  of  the  tests  will  be 
scored  as  events  for  the  binomial  distribution.  The  sample 
data  will  be  fit  to  the  distributions  using  a  Komolgorov- 
Smirnov  (KS)  Goodness  of  Fit  test. 

The  Classical  Exponential  Approach.  Assuming  the  data 
are  distributed  exponentially,  point  estimates  for  the 
failure  rate,  ,  and  a  90%  confidence  interval  about  A  will 
be  calculated. 

Classical  Exponential  Point  Estimate .  The  maximum 
likelihood  estimator  for  the  exponential  distribution  is  R/T 
where  R  is  the  number  of  failures,  presumed  to  be  a  random 
variable,  and  T  is  the  total  time  on  test. 
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Classical  Exponential  Confidence  Interval.  The 

confidence  limits  for  the  exponential  distribution  may  be 

.  2  2 
approximated  using  the  X  (n)  distribution  where  X  (n)  is 

A 

p 

the  100(l-c()  percentile  of  the  X  (n)  distribution. 

In  the  case  of  Type  I /time  truncated  testing  with 
replacement,  the  sufficient  statistic  R  (the  random 
number  of  failures)  follows  a  Poisson  distribution-... 
and  the  relationship  between  the  Poisson  and  the  Yr  (n) 
distributions  permits  approximate  inferences  to  be 
based  on  the  Y7  (n)  distribution.  (22:120-1) 

The  100(1-*)%  TCI  for  (22:122)  is 


X2^/2(2R)  X21_s</2(2R+2) 


The  Binomial  Approach .  The  binomial  distribution  has  a 
maximum  likelihood  estimator  p'  such  that  the  probability  of 
survival  is  the  ratio  of  X  survivors  from  a  set  of  n  units 
placed  into  a  test  of  given  duration  (22:53). 


(11) 


The  100(1-*)%  TCI  may  be  approximated  using  a 
transformation  to  the  F  distribution  (22:56)  given  by  the 
formula 

x 

x+  (n-x+1)  F1_  ^2  (2n-2x+2, 2x) 
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(x+1)F1_#^2  (2x+2,2n-2x) 
(n-x)  +  {x+l)F^_fl^2  (2x+2, 2n-2x) 


(12) 


Bayesian  Models 

The  Data  Sources .  As  a  weapon  system  is  transitioning 
from  development  to  production,  there  is  both  a  wealth  and 
paucity  of  available  data  upon  which  to  base  reliability 
predictions.  Generally,  the  engineering  department  is  long 
on  analyses  and  comparisons  to  existing  systems,  while  the 
test  department  has  little  in  the  way  of  operational  data; 
being  limited  chiefly  to  testing  with  engineering 
prototypes.  There  is  little  disagreement  that  neither  of 
these  by  themselves  constitute  a  data  base  with  high 
validity.  It  is  possible  to  identify  other  data  sources 
which  when  compiled  in  a  model  can  produce  a  more 
comprehensive  picture  of  component  reliability.  These 
include: 

1)  operational  system  level  test  data, 

2)  system  level  acceptance  testing, 

3)  component  level  acceptance  testing, 

4)  environmental  stress  screening  data, 

5)  qualification  testing  data, 

6)  design  analyses,  such  as  Mil-Hdbk-217D, 

7)  comparisons  to  preceding  weapon  systems. 

The  conservative  application  of  the  Bayesian  approach 
uses  old  flight  test  data  to  develop  the  prior.  In  the 
event  of  insufficient  flight  test  data,  system  level  and 
possibly  box  level  acceptance  test  data  could  be  used. 
Acceptance  testing  may  not  be  representative  of  the 
operational  environment,  and  the  objective  use  of  these 
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types  of  test  data  can  introduce  additional  risk  to  the 
analysis . 


The  Bayesian  Exponential  Approach.  Assuming  an 
exponentially  distributed  failure  rate,  the  distribution  of 
failures  in  a  fixed  total  test  time  can  be  described  by  the 
Poisson  distribution  (22:255). 

e‘^*(Mt)  s 

p(s  failures  in  total  time  1 1  A)  -  (13) 

s ! 

where,  A,t  >  0,  s  =  0,  1,  2,  ....  Because  not  all  of  the 
flight  test  units  failed  during  the  test  and  failed  units 
were  replaced  by  new  units  for  subsequent  tests,  the  flight 
test  data  may  be  considered  to  be  Type  I,  time  truncated, 
with  replacement  (22:119). 

Bayesian  Exponential  Prior.  This  Poisson  sampling 
process  can  use  a  gamma  pri.or  distribution  to  achieve 
maximum  flexibility  and  ease  the  mathematical  burden  of 
calculating  the  posterior  distribution.  Because  of  these 
two  reasons,  the  gamma  prior  distribution  (22:289)  is 

one  of  the  most  widely  used  for  A,  and  has  the  following 
probability  density  function: 

g U;a,0  )  - - *  V  *  e  <14) 

P  W) 

where  A, *  ,  and  £  >  0  . 
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The  shape  parameter,  * ,  and  the  scale  parameter, $  ,  may 
be  interpreted  as  the  "pseudo  number  of  failures"  and 
"pseudo  length  of  time"  for  the  prior  life  test.  The 
"pseudo"  refers  to  the  supposition  that  these  are  the  values 
that  would  best  fit  a  prior  appropriate  for  the  experimental 
data  (22:289) . 

The  simplest  estimates  for  &  and  $  are  the  number  of 
flight  test  failures  and  total  test  time. 

Bayesian  Exponential  Preposterior  Analysis .  A 
preposterior  analysis  will  be  conducted  using  two 
hypothetical  samples  of  data.  The  first  set,  considered 
likely,  is  one  failure  in  256  hours,  the  CMGS  MTBF  (29:1D). 
The  unlikely  data  set  consists  of  ten  failures  in  256  hours. 
Posteriors  for  each  will  be  calculated  using  the  gamma  («*,/ 3  ) 
prior  distribution  and  evaluated  for  realism.  The  prior 
distribution  will  be  adjusted  if  necessary. 

Bayesian  Exponential  Posterior.  The  resulting 
posterior  distribution  for  the  gamma  (<*.,0 )  prior  distribution 
is  gamma  (s+*,  t+(5)  ,  where  s  and  t  are  the  number  of  failures 
and  total  test  time  from  the  CMGS  Reliability  Demonstration. 
The  probability  density  function  (22:290)  is  described  by 


g  ( Al  ) 


(t+3)s+* 

r<sM) 


*  Xs+*-1*  e” (t+^)^ 


(15) 


where  \  >  0 . 

Bayesian  Exponential  Point  Estimate .  The  Bayesian 
point  estimator  is  the  mean  of  the  posterior  distribution, 


38 


which,  under  the  squared  error  loss  function,  is 


s  +  * 

E(Als;<*,S)  =  - -  (16) 

t 

where  s  +  is  the  combined  number  of  failures  and  t  +  P  is 

the  combined  total  test  time  (22:292). 

Bayesian  Exponential  Probability  Interval .  The 

Total  Bayesian  Probability  Interval  (TBPI)  can  be 

.  2 

constructed  by  using  chi  square  (X  )  percentile  values  for 
(2s  +  2*) .  A  90%  confidence  interval  will  be  calculated  for 
comparison  with  intervals  from  the  other  models.  The  upper 
and  lower  interval  endpoints  for  the '  100  (1~<)  %  TBPI  (22:294) 
are 


-Imill.I.l'l  .  *hz*Ll{ls---±?-  (17) 

2  (t  +  &  )  '  2  (t  +  fi  ) 

The  Bayesian  Binomial  Approach.  For  the  binomial  case, 
the  family  of  £{x,n)  distributions  will  be  used  to  calculate 
both  the  prior  and  posterior  distributions  because  they  are 
relatively  easy  to  calculate  and  may  be  adapted  to  a  number 
of  situations.  For  example,  the  selection  of  x  and  n  can  be 
based  on  the  analyst's  knowledge  and  experience  with  the 
system.  This  can  have  the  effect  of  increasing  or 
decreasing  the  importance  of  the  prior  on  the  resulting 
posterior  distribution. 
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Bayesian  Binomial  Prior.  The  £(xQ>ng) 

distribution  is  a  conjugate  prior  for  the  binomial  sampling 

model.  The  choice  of  Xg,  the  "pseudo  number  of  survivors," 

and  ng,  the  "pseudo  sample  size,"  can  be  made  on  the  basis 

of  the  analyst's  prior  knowledge,  or  based  on  the  results  of 

earlier  objective  testing. 

Weiler  (1965)  shows  that  the  effect  of  assuming  a 
^}(nQ,x„)  distribution,  when  in  fact  the  true  prior 
distribution  is  not  of  the  beta  type,  is  negligible  in 
many  practical  applications.  He  shows  that  rather 
severe  deviations  in  the  beta  prior  parameter  values 
produce  only  slight  changes  in  the  corresponding 
posterior  distributions.  (22:265) 


For  this  research,  earlier  flight  test  data  will  be 
used  to  estimate  values  for  Xg  and  nQ  that  fit  the  ^(Xg,ng) 
distribution.  The  simplest  estimate  for  xQ  is  the  number  of 
CMGS  units  that  have  successfully  flown  in  flight  testing. 
Because  there  was  a  significant  configuration  change  in  the 
CMGS  computer,  only  flight  test  data  pertaining  to  the  newer 
configuration  will  be  used  to  develop  this  prior 


distribution.  Similarly,  the  simplest  estimate  for  ng  is 
the  number  of  guidance  sets  that  have  been  flight  tested. 

Bayesian  Binomial  Preposterior  Analysis .  As  above 
for  the  Bayesian  exponential  approach,  a  preposterior 
analysis  will  be  conducted  using  two  hypothetical  data  sets. 
The  likely  sample  will  consist  of  99  successes  out  of  100 
attempts  and  the  unlikely  sample  will  be  90  successes  out  of 
100  attempts.  An  evaluation  of  the  prior  will  be  made  based 
on  these  results. 
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Bayesian  Binomial  Posterior.  The  resulting 
posterior  distribution  is  also  a  beta  distribution  of  the 
form  (x+XQ,n+nQ),  where  x  and  n  are  the  observed  survivors 
and  sample  size  from  the  experiment  being  used  to  update  the 
prior.  In  this  case,  x  and  n  will  be  the  number  of  missions 
successfully  completed  and  the  number  attempted, 
respectively.  The  probability  density  function  (22:266)  of 
the  new  posterior  distribution  is 


g(p|x;xQ/n0) 


P  (n  +  nQ) 

r<x  +  Xq)  f  (n  +  nQ  -  x  -  xQ) 


(x+xn)-l  (n+nn-x-xn) -1 

P  *  (1-P)  U  U  (18) 


where  0  <  p  <  1 . 

Bayesian  Binomial  Point  Estimate.  A  point 
estimator  for  the  probability,  p,  which  represents 
reliability  in  this  sampling  scenario,  can  be  determined 
from  the  posterior  mean  (22:267). 


E(P|x;x0,nQ)  = 


x  +  x 


n  +  n 


0 


0 


(19) 


Bayesian  Binomial  Probability  Interval .  Again 
assuming  a  $(xQ,ng)  prior,  the  Total  Bayesian  Probability 
Interval  (TBPI)  can  be  constructed  from  a  transformation  to 
the  F(nlfn2)  distribution.  A  90%  confidence  interval  will 


41 


be  calculated  for  comparison  with  intervals  from  the  other 
models.  The  upper  and  lower  interval  endpoints  (22:270)  for 
the  100(1-*)%  TBPI  are 

x  +  xQ 

—  — —  —  —  —  —  —  —  —  —  —  —  —  ———————  f 

x+Xq  +  (n+nQ-x-Xg)  f1_^2  (2n+2riQ-2x-2x0,  2x+2xQ) 

_^t^0^1-*/2^x+^x0/  2n+2n0-2x-2x0)^ 
n+nQ-x-XQ+ (x+xc)F1_^/2 (2x+2xq,  2n+2n3~2x~2xQ) 

The  Expert  Information  Approach.  The  Kaplan  model 
described  earlier  will  be  used  with  some  of  the  above  data 
sources  to  make  the  most  accurate  reliability  prediction 
possible.  The  chief  advantage  in  using  Bayes'  Theorem  in  a 
predictive  model  is  its  abil ity  to  incorporate  data  normally 
excluded  as  being  "subjective"  in  classical  approaches. 

This  data  is  analyzed  and  the  appropriate  subjective 
probability  distribution  (SPD)  describing  the  data  is  used 
to  develop  the  prior  distribution.  The  SPD  is  a  discrete 
distribution  whose  total  area  sums  to  one.  Expert 
information  can  be  used  to  evaluate  each  of  the  data  sources 
used  in  preparing  and  combining  the  prior  distributions  to 
form  the  final  prior  distribution. 

Both  the  exponential  and  binomial  approaches  will  be 
investigated  using  the  following  Bayesian  relationships: 

P(XIA.) 

P(^.  IX)  =  p(A.)  - (21) 

P(X) 
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where, 

p(^.  IX)  =  probability  of  failure  rate  given  the 
1  experience  X 

p(\.)  =  probability  of  A^,  prior  to  having  X 
p (X)  =  prior  probability  of  X,  also 

=  S  p (X) p (X | X. ) ,  for  all  discrete  values  of  Ai 
*i  1 

for  the  exponential: 

p(X|Ai)  =  probability  of  X,  for  each  given  A^, 

(A.  *  T)n 

- - 1 - *  exp  (-A.  *T) , 

n! 

with  n  failures  in  time  T, 
for  the  binomial: 

p(X|Ai)  =  probability  of  X,  for  each  given  Xi, 

= - -- - *  (A.)k(l-Ai)n_k 

k!  *  (n-k) ! 

with  k  failures  in  n  attempts. 

Bayesian  Expert  Information  Prior.  Using  the 
definitions  of  evidence  presented  by  Kaplan  in  Table  2,  the 
following  data  will  be  used  to  develop  the  iterative  priors. 

Step  1.  Type  E8  -  The  computer  used  in  the  CMGS  is  its 
most  complex  component  and  bears  the  greatest  allocated 
reliability  requirement.  It  is  also  very  similar  to  the 
Inertial  Navigation  Element  used  in  the  Air  Launched  Cruise 
Missile  for  the  past  six  years.  A  subjective  evaluation  of 
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the  CMGS  reliability  will  be  based  on  a  comparison  of  ALCM 
INE  field  performance  data  (35)  to  its  reliability 
specification.  For  example,  if  the  ALCM  INE  has  a 
demonstrated  MTBF  of  500  hours  versus  a  specification  value 
of  400,  then  the  probability  distribution  for  the  CMGS  could 
be  adjusted  upward  from  its  specification  value  of  300  to  a 
value  25%  higher. 

The  specific  amount  of  adjustment  will  be  based  on  an 
estimate  provided  by  Mr.  I.  Hugh  Lynn,  the  former  GLCM  Chief 
Avionics  Engineer.  He  was  responsible  for  conducting 
technical  evaluations  during  the  CMGS  Reliability 
Demonstration.  Mr.  Lynn  has  been  a  practicing  government 
engineer  for  the  past  22  years  and  is  currently  the  Computer 
Resources  Branch  Chief  (ASD/ENASC)  for  the  Aeronautical 
Systems  Division,  Wright  Patterson  AFB,  OH.  His  experience 
is  typical  of  Department  of  Defense  individuals  who  would  be 
tasked  to  make  this  type  of  an  evaluation  for  current  weapon 
systems  (21) . 

Issues  to  be  considered  include  how  many  components  in 
the  CMGS  do  not  have  comparable  functions  in  the  INE  and  how 
the  ALCM  and  Tomahawk  cruise  missile  environments  compare. 
The  MTBF  estimates  will  be  converted  to  failure  rates  to 
make  the  comparisons  more  tractable.  Because  there  is 
uncertainty  in  these  estimates,  a  range  of  failure  rates 
with  associated  probabilities  can  be  plotted  to  form  the 
SPD .  This  distribution  is  the  First  Prior  (see  Figure  5) . 
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Figure  o.  Subjective  Probability  Distribution  (SPD) 
for  the  CMGS  Based  on  ALCM  INE  Field  Data 


Step  2.  Typa  E6  -  CMGS  design  weaknesses  identified  in 
the  Ground  Launched  Cruise  Missile  Flight  Critical  Item 
Investigation  (11)  are  the  basis  for  updating  the  First 
Prior  distribution.  The  informed  source  (Mr.  Lynn) 
evaluates  the  impact  of  each  weakness  on  the  designed 
reliability  to  develop  a  likelihood  that  can  be  used  to 
update  the  First  Prior.  The  resulting  posterior 
distribution  is  used  as  the  Second  Prior  for  the  next 
iteration  involving  simulated  testing. 

Step  3.  Type  E2  and  E4  -  Performance  of  the  CMGS  in 
acceptance  testing  and  failure  free  Environmental  Stress 
Screening  (ESS)  testing  is  used  to  update  the  Second  Prior 
to  reflect  the  performance  of  the  CMGS  in  sim  alated 
environments.  Additionally,  CMGS  performance  in  early  ESS 
cycles  will  be  evaluated  as  a  lower  bound  on  MTBF .  This  is 
due  to  the  presence  of  manufacturing  defects  that  are 
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precipitated  as  failures  in  early  ESS  temperature  and 
vibration  cycles.  The  design  posterior  distribution  is 
updated  by  this  simulated  testing  SPD  to  create  the  Third 
Prior. 

At  this  point,  the  analysis  will  independently  pursue 
both  exponential  and  binomial  approaches  to  continue 
developing  the  final  priors  and  estimate  CMGS  failure  rates. 
The  SPD  used  for  the  Third  Prior,  as  well  the  resulting 
posterior  distributions  will  be  updated  using  Bayes'  Theorem 
and  the  Poisson  model  for  the  exponential  approach.  For  the 
binomial  approach,  Bayes'  Theorem  and  equation  (14)  will  be 
used. 

Step  4a.  Type  El  -  (Exponential)  -  The  Third  Prior 
distribution  is  updated  by  flight  test  results  to  form  the 
Final  Prior.  The  flight  test  database  will  be  examined  to 
ensure  only  flight  data  from  comparable  CMGS  configurations 
is  included  in  the  analysis.  These  configurations  are 
defined  as  those  guidance  sets  manufactured  in  the  fiscal 
year  FY  1983  and  since  then.  The  number  of  failures  and 
total  mission  time  will  be  used  to  obtain  probabilities  l~r 
each  failure  rate,  A^.  These  probabilities  form  the  Final 
Prior. 

Step  4b.  Type  El  -  (Binomial)  -  The  Third  Prior  is 
similarly  updated  by  flight  test  results  in  the  form  of  the 
numbers  of  successes  and  attempts.  Equation  (14)  will  be 
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used  to  obtain  probabilities  for  each  that  will  form  the 
Final  Prior. 

Bayesian  Expert  Information  Posterior .  The 
Final  Priors  will  be  used  to  obtain  posterior  SPD's  for  each 
of  the  three  phases  of  the  CMGS  Reliability  Demonstration 
and  for  a  combination  of  all  three  phases.  Mission  time 
will  be  assumed  to  be  a  standard  three  hour  flight  test  time 
for  use  in  calculating  the  reliability  according  to  equation 

(D  ,  rcmgs  =  exP<_3*A>  •  • 

Step  5a.  (Exponential)  -  The  number  of  CMGS  Reliability 
Demonstration  test  hours  is  adjusted  by  a  series  of  k- 
f actors  to  allow  comparison  to  the  standard  mission  time. 
This  research  accepts  the  use  of  the  specified  k-f actors 
without  examination.  The  number  of  failures  is  used  with 
the  total  test  time  to  calculate  the  posterior  distribution 
based  on  the  Final  Prior  distribution.  This  posterior 
distribution  will  be  evaluated  to  determine  a  point  estimate 
for  A  and  a  Total  Bayesian  Probability  Interval  (TBPI) . 

Step  5b.  (Binomial)  -  The  binomial  approach  will  focus 
on  the  number  of  successful  missions  that  were  accomplished 
out  of  the  total  number  attempted.  The  Final  Prior  from 
Step  4.b  above  will  be  updated  by  this  information  to  form 
the  resulting  posterior  distribution.  As  in  Step  5. a,  both 
a  point  estimate  and  a  TBPI  will  be  determined  for  A. 

Bayesian  Expert  Information  Point  Estimate  and 
Probability  Interval .  The  resulting  posterior  distributions 
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will  be  evaluated  to  obtain  both  point  estimates  for  the 
failure  rate,  \  ,  and  the  90%  Total  Bayesian  Probability 
Interval  (TBPI) . 

Step  6.  The  point  estimate  is  the  expected  value  of  , 

E(A)  =  1  P(A.)  *  A.  (22) 

i 

and  the  TBPI  is  the  interval  that  encompasses  90%  of  the 
area  under  the  SPD  while  maximizing  the  sum  of  the 
probability  densities  associated  with  each  point  in  the 
interval . 

The  methodology  used  to  develop  each  of  the  six  models 
will  be  applied  to  the  CMGS  Reliability  Demonstration  data 
and  the  results  will  be  described  in  Chapter  IV.  The  six 
model  predictions  will  be  compared  with  current  Tomahawk 
flight  test  data  to  ascertain  the  accuracy  of  each  model. 

An  evaluation  of  both  the  approaches  (Classical  versus 
Bayesian)  and  the  choice  of  distributions  (exponential 
versus  binomial  versus  expert  information)  will  be  based  on 
1)  how  the  point  estimators  compare  to  the  flight  test  mean 
value  and  2)  the  width  of  the  confidence  or  probability 
intervals  for  each  of  the  models. 
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IV.  Results 


Overview 

The  results  of  the  calculations  described  in  the 
methodology  of  Chapter  III  will  be  described  and  discussed 
in  this  section.  Most  of  the  calculations  will  be  shown  in 
detail  the  first  time  presented  and  referenced  in  Appendix  F 
for  subsequent  applications.  The  flight  test  data  will  be 
examined  first  to  establish  a  benchmark  with  which  to  judge 
the  prospective  models.  Point  estimates  and  confidence 
intervals  will  be  calculated  in  the  order  presented  in  the 
methodology,  beginning  with  the  classical  models,  continuing 
through  the  Bayesian  distributions,  and  completing  with  the 
Bayesian  expert  information  approach.  This  will  be  followed 
by  a  comparison  of  each  of  the  model's  predictions  with  the 
flight  test  values  determined  earlier.  The  flight  test 
classical  exponential  and  binomial  approaches  will  be 
addressed  next. 

Flight  Test 

The  flight  test  data  (see  Appendix  A)  was  evaluated 
using  the  methods  of  classical  statistical  inference.  The 
database  (26)  contained  data  from  all  134  Tomahawk  Land 
Attack  Missile  flight  tests,  representing  a  number  of 
different  configurations.  In  order  to  have  enough  data 
points  to  estimate  the  failure  distribution,  a  Goodness-of- 
Fit  test  was  conducted  using  failure  data  from  the  complete 


database.  Reliability  improvements  were  incorporated  in 
some  FY83  production  units  and  in  most  FY84  and  later  units. 
Point  estimates  and  confidence  intervals  were  determined  for 
three  samples  from  the  database:  a)  all  flight  tests,  b) 

FY83  and  on,  and  c)  FY84  and  on.  Results  of  the  Kolmogorov- 
Smirnov  test  will  be  reviewed  next  to  see  if  the  flight  test 
failures  were  distributed  exponentially. 

Goodness -of -Fit  Test .  Using  the  sample  point  estimate 
for  derived  from  the  all  of  the  flight  test  failures,  an 
exponential  distribution  of  random  points  was  compared  with 
the  actual  flight  times.  The  results  of  the  test  are  shown 
in  Figure  6 . 

The  p-value  for  this  test  indicates  that  the  two 
distributions  are  not  statistically  distinct.  This  implies 
that  the  CMGS  flight  test  failures  are  exponentially 
distributed.  Because  only  two  flight  test  failures  have 
occurred  with  the  FY83  and  later  configurations,  all  CMGS 
flight  failures  have  been  included  for  the  Kolmogorov- 
Smirnov  Test.  Caution  must  be  used  in  interpreting  these 
results  because  of  the  small  sample  size  and  varying 
configurations  of  earlier  flight  test  units.  The  point 
estimates  and  confidence  intervals  were  calculated  using  all 
the  flight  test  operating  time,  including  those  units  that 
flew  successfully.  As  such,  the  flight  tests  were  regarded 
as  a  Type  I,  time  truncated  tests  were  directly  comparable 
to  the  CMGS  Reliability  Demonstration  results. 
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TWO  SAMPLE  KOLMOGOROV- SMIRNOV  (SMIRNOV)  TEST 

1.0  I 


o*o  ^ _ 


-  TIME  ===  TIME2 

SAMPLE  SIZE  FOR  TIME  9 

SAMPLE  SIZE  FOR  TIME2  11 

HYPOTHESIS  :  TIME  <>  TIME2 

TWO  TAILED  KOLMOGOROV-SMIRNOV  STATISTIC  0.12 

HYPOTHESIS  :  TIME  <  TIME2 

ONE  TAILED  KOLMOGOROV-SMIRNOV  STATISTIC  0.00 

HYPOTHESIS  :  TIME  >  TIME2 

ONE  TAILED  KOLMOGOROV-SMIRNOV  STATISTIC  0.12 


Figure  6.  Flight  Test  Data  Kolmogorov-Smirnov  Test 
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The  number  of 


The  Classical  Exponential  Approach, 
failures  and  total  operating  time  for  the  flight  tests  is 
shown  in  Table  3.  The  point  estimate  for  the  failure  rate 
is  the  number  of  failures  divided  by  the  total  operating 
time  in  the  test. 

Classical  Exponential  Confidence  Interval .  The 
90%  confidence  interval  about  the  point  estimate  was 
obtained  for  the  FY83  data  using  a  chi  square  transformation 
with  equation  (11) . 


Lower 

1.635 

Limit 

= - 

=  .01087 

2  *  75.2 

Upper 

9.488 

Limit 

2  *  75.2 

=  .06309 

The  Binomial  Approach.  This  approach  used  the  number 
of  successful  missions  completed  divided  by  the  number 
attempted  to  develop  a  point  estimate  for  p,  the  probability 
of  success.  This  value  was  then  converted  to  a  failure  rate 
using  the  exponential  relationship  described  in  equation 
(1) .  The  binomial  point  estimates  and  confidence  intervals 
for  the  flight  test  data  are  shown  in  Table  3. 

Classical  Binomial  Confidence  Interval .  The  90  % 
confidence  interval  about  the  point  estimate  was 
approximated  using  a  transformation  to  the  F  distribution  as 
described  in  equation  (12) .  For  the  flight  test  data, 


Table  3.  Flight  Test  Results 


Exponential 


Data 

Time 

(T) 

Failures 

(R) 

A 

(r7t) 

Confidence 

Lower 

Limit 

Upper 

All 

176.441 

12 

.06801 

.04358 

.10321 

FY83 

75.2 

2 

.02660 

.01087 

.06309 

FY84 

54.65 

★ 

1 

.01830 

.006502 

.05481 

Binomial 

Data  Number  of  Missions  P  A  Confidence  Limit 
Success  Attempt  (X/N)  Lower  Upper 


(X) 

(N) 

All 

107 

119 

.8992 

.03543 

.02301 

.05253 

FY83 

48 

50 

.9600 

.01361 

.003657 

.03610 

FY84 

41 

★ 

42 

.9762 

.008033 

.0007849 

.03118 

* 

One 

failure 

assumed 

to  occur 

on  next 

mission. 

t  =  0 

Lower  (48  +  1) *F  q_  (2*48+2,  2*50-2*48) 

Limit  - - - - 

50-48+ (48  +  1) *F  g5  (2*48+2,  2*50-2*48) 

49*F  9  (98,  4)  49*3.7 

= - +22 -  =  -  =  .9891 

2  +  49*F  9 5  (98,  4)  2+49*3.7 

and  converting  to  a  failure  rate,  R(t)  =  exp(-A*3), 

=  In  (. 9891) /-3  =  .003657 


Upper  48 

Limit  =  - 

48+ (50-48  +  1) *F  95  (2*50-2*48+2,  2*48) 
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.8974 


48  48 

48  +  3*F  g5  (6,  96)  48+3*1.83 

=  In (R) / (-3)  =  In ( . 8974) /-3  =  .03610 

The  differences  in  the  flight  test  A  point  estimates 
and  confidence  intervals  between  the  exponential  and 
binomial  approaches  raise  concerns  over  the  use  of  this 
flight  test  data  as  a  benchmark  for  model  performance.  If 
the  actual  failure  rate  A  was  exponentially  distributed, 
then  based  on  the  model  mission  time,  the  binomial  and 
exponential  values  should  have  been  equivalent. 

The  configuration  of  the  CMGS  units  had  an  impact  on 
the  demonstrated  flight  test  reliability.  The  FY84 
configuration  was  probably  most  representative  of  the 
current  CMGS  configuration,  however,  because  the  point 
estimates  and  confidence  intervals  for  the  failure  rate  were 
derived  by  including  a  hypothetical  failure,  the  true  A  for 
this  population  was  probably  lower  than  indicated  above. 
Lacking  sufficient  data  for  the  FY84  configuration,  the  FY83 
configuration  data  was  used  as  a  prior  for  the  Bayesian 
approaches . 

The  Classical  Models 

As  described  in  the  methodology,  both  exponential  and 
binomial  approaches  were  used  with  the  CMGS  Reliability 
Demonstration  data  (28-30)  from  each  of  the  phases  of 
testing.  A  prediction  for  the  point  estimate  and  a 
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corresponding  confidence  interval  was  made  for  a  combination 
of  all  of  the  data  as  well. 


The  Classical  Exponential  Approach.  The  number  of 
failures  and  total  operating  time  in  each  test  is  shown  in 
Table  4  for  each  of  the  test  phases.  The  point  estimate  for 
the  failure  rate  A  was  the  number  of  failures  divided  by  the 
total  operating  time  in  the  test. 

Classical  Exponential  Confidence  Interval .  The 
90%  confidence  interval  about  the  point  estimate  was 
obtained  using  a  chi  square  transformation  with  equation 
(11)  • 


Lower  2.733 

Limit  = - -  .001444 

2  *  946.846 

Upper  12.592 

Limit  - - -  .006648 

2  *  946.846 


Table  4.  Classical  Exponential  Model 


Test 

Time 

(T) 

Failures 

(R) 

A 

(R/T) 

Confidence 

Lower 

Limit 

Upper 

Phase  I 

946.486 

3 

.003170 

.001444  . 

006648 

Phase  II 

423.11 

1 

.002364 

.0008399  . 

007080 

Phase  I 

881.413 

3 

.003404 

.001550  . 

007)42 

Extension 

Combined 

2251 

7 

.003110 

.001769  . 

005260 
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The  Binomi al  Approach.  This  approach  used  the  number 
of  successful  missions  completed  divided  by  the  number 
attempted  to  develop  a  point  estimate  for  p,  the  probability 
of  euccess.  This  value  was  then  converted  to  a  failure  rate 
using  the  exponential  relationship  described  in  equation 
(1) .  The  data  and  resulting  values  for  the  CMGS  Reliability 
Demonstration  test  phases  are  shown  in  Table  5. 

Table  5.  Classical  Ednomial  Model 

Test  Number  of  Missions  X/N  h  Confidence  Limit 
Success  Attempt  Lower  Upper 


(X) 

(N) 

Phase  I 

279 

282 

.9894 

.003565 

.001310  .007893 

Phase  II 

125 

126 

.9921 

.002656 

.0002787  .01034 

Phase  I 
Extension 

261 

264 

.9886 

.003810 

.00140  .008424 

Combined 

665 

672 

.9896 

.003490 

.001941  .005922 

Classical  Binomial  Confidence  Interval .  The  90  % 
confidence  interval  about  the  point  estimate  was 
approximated  using  a  transformation  to  the  F  distribution  as 
described  in  equation  (12)  . 

Classical  Binomial  Confidence  Interval.  The 
confidence  interval  was  determined  using  equation  (12)  and 
the  F  distribution  for  1-  /2  =  .95.  For  the  Phase  I  data. 

Lower  (279  +  1) *F  q,  (2*279+2,  2*282-2*279/ 

Limit  =  - - - - - - 

282-279+ (279+1) *F  g5(2*279f 2,  2^282-2*279) 
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.9961 


280*F  95  (560,  6)  280*2.72 

3+280*F  95(560,  6)  3+280*2.72 

and  converting  to  a  failure  rate,  R(t)  =  exp(-X*3), 

=  In  ( . 9961) /-3  =  .001310 

Upper  279 

Limit  - - - - 

279+ (282-27  9+1) *F  ?5  (2*282-2*279+2,  2*279) 

279  279 

= - =  -  o  .9765 

(279+4*F  95(8,558)  279+4*1.68 

*  ln(R)'(“3)  =  in  ( . 9765) /-3  =  .007934 

There  was  good  agreement  between  the  classical 
exponential  and  binomial  point  estimates  for  X .  In  both 
cases,  they  were  considerably  lower  than  the  flight  test 
failure  rates,  and  had  confidence  intervals  enclosing  the 
point  estimates  '".hat  were  an  order  of  magnitude  smaller  than 
the  flight  test  confidence  intervals.  The  binomial 
confidence  intervals  were  wider  than  those  for  the 
exponential  approach. 

The  Bayesian  Models 

The  next  sections  describe  the  results  obtained  lrom 
the  Bayesian  exponential,  binomial  and  expert  information 
models,  the  last  having  both  exponential  and  binomial 
approaches.  In  the  development  of  each  model,  a  sequence  of 
steps  was  followed: 
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1)  Develop  prior  distribution, 


2)  Calculate  posterior  distribution  or  iterative 
priors, 

3)  Determine  Bayesian  point  estimate  for  failure  rate, 

4)  Determine  Total  Bayesian  Probability  Interval. 

The  Bayesian  exponential  model  was  examined  first. 

Bayesian  Exponential  Model .  This  model  used  flight 
data  from  comparable  'CMGS  configurations  to  develop  the 
gamma  (<*,0  )  prior  distribution.  The  prior  was  then  updated 
by  results  from  the  CMGS  Reliability  Demonstration  to 
predict  point  estimates  and  TBPI's  for  each  of  the  three 
test  phases  and  the  combination  of  all  three. 

Bayesian  Exponential  Prior.  For  the  purposes  of 
this  research,  flight  test  data  from  CMGS  units  built  in 
FY83  ana  later  production  lots  was  used  to  develop  the  prior 
distribution.  As  described  in  the  methodology,  the  simplest 
estimate  for  the  and  &  parameters  of  the  gamma 
distribution  were  the  number  of  flight  test  failures,  s, 
and  total  test  time,  t.  The  gamma (2, 75 . 2)  prior 
distribution,  based  on  the  flight  test  data,  was  used  to 
obtain  the  posterior  distribution. 

Bayesian  Exponential  Preposterior  Analysis .  The 
prior  was  evaluated  using  two  hypothetical  samples  of  data. 
The  likely  data  was  one  failure  in  256  hours  and  resulted  in 
the  following  point  estimate: 
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s  +  t*.  1  +  2 

-  =  -  =  .009058 

t  +  fi  256  +  75.2 

The  unlikely  data  was  ten  failures  in  the  same  256  hours  for 
a  point  estimate  of 

s  +  eC  10  +  2 

-  =  -  =  .03623 

t  +0  256  +  75.2 

These  values  indicated  that  the  gamma (2, 75 . 2)  prior  was 
performing  in  a  proper  manner.  If  the  point  estimates  for 
these  two  data  samples  had  been  similar,  the  prior  would 
have  been  adjusted. 

Bayesian  Exponential  Posteriors .  The  gamma (2, 75.2) 
prior  was  updated  by  the  CMGS  Reliability  Demonstration 
failures,  s,  and  total  time,  t,  in  test  for  each  of  the 
three  phases  and  the  combination  of  all  three.  The  values 
for  s  and  t  are  tabulated  in  Table  6  along  with  the  point 
estimates  and  TBPI's  for  the  new  gamma (s+*, t+fi)  posterior 
distributions . 

Bayesian  Exponential  Point  Estimate.  Equation 
(16)  defines  the  Bayesian  point  estimator  for  the  mean.  The 
calculation  for  Phase  I  data  was 

s  +  (A  3  +  2 

-  =  -  =  .004894 

t  +  0  946.486  +  75.2 
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.4* 


Bayesian  Exponential  Probability  Interval .  The 
TBPI  for  each  of  the  CMGS  Reliability  Demonstration  test 
phases  was  calculated  using  chi  square  percentile  values. 
Phase  I  calculations  are  shown  below,  based  on  equation 
(17),  and  the  remaining  values  are  tabulated  in  Table  6. 


Lower 
Limit  = 


X^  95(2*3  +  2*2) 


3.94 

2043.37 


=  .001928 


2*  (946.486  +  75.2)  2043.37 

Table  6.  Bayesian  Exponential  Model 


Test 

Time  #  of 

Failures  s+«* 

Confidence  Limit 

(T) 

(R) 

t+0 

Lower 

Upper 

Phase  I 

946.486 

3 

.004894 

.001928 

.008961 

Phase  II 

423.11 

1 

.006020 

.001641 

.01263 

Phase  I 

881.413 

3 

.005338 

.002106 

.009775 

Extension 

Combined 

2251 

7 

.003869 

.002018 

.006205 

Upper  X  ns(2*3  +  2*2)  18.31 

Limit  =  — -  = -  =  .008961 

2*  (946.486  +  75.2)  2043.37 


Bayesian  Binomial  Model .  For  the  binomial  approach, 
the  ^3(x,n)  distribution  was  used  for  both  the  prior  and 
posterior  distributions.  X  was  the  number  of  successful 
missions  accomplished  and  n  was  the  number  attempted.  As  in 
the  Bayesian  exponential  model,  the  prior  was  based  on 
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relevant  flight  test  data  and  updated  with  the  results  of 
the  CMGS  Reliability  Demonstration, 

Bayesian  binomial  Prior.  The  /2(XQ,ng) 
distribution  was  used  for  the  prior,  where  x^  was  the  number 
of  flight  test  successes  and  ng  was  the  total  number  of 
flights  attempted.  As  in  the  Bayesian  exponential  model, 
only  flights  using  CMGS  units  produced  in  FY83  and  later 
production  lots  were  scored.  ' 

Bayesian  Binomial  Preposterior  Analysis .  The 
preposterior  analysis  for  this  approach  used  binomial 
hypothetical  data.  The  likely  data  was  99  successes  out  of 
100  attempts  with  a  point  estimate  of: 


x  +  x. 


99  +  48 


- =  - -  .9800 


n  +  n. 


100  +  50 


convert  to  A,  A  =  ln(.9800)/-3  =  .006734 


The  unlikely  data  was  90  successes  out  of  100  attempts  with 
a  point  estimate  of: 


x  +  x. 


n  +  n. 


90  +  48 

-  =  .9200 

100  f  50 


convert  to  A, 


A  =  In  ( . 9200) /-3  =  .02779 


The  results  of  this  analysis  again  indicated  that  the 
prior  was  functioning  properly,  as  evidenced  by  the  wide 
difference  between  the  two  failure  rates. 


61 


Bayesian  Binomial  Posterior.  The  $(48,50)  prior 
was  updated  with  the  results  of  the  CMGS  Reliability 
Demonstration  to  form  four  new  posterior  $(x+Xg,n+ng) 
distributions  (including  the  combination  of  phase  data) . 

The  x  and  n  parameters  were  the  successes  and  total  attempts 
for  each  of  the  test  phases. 

Bayesian  Binomial  Point  Estimate .  The  point 
estimator  for  the  reliability  was  calculated  for  Phase  I 
data  using  equation  (19) .  The  results  of  point  estimate  and 
TBPI  calculations  for  other  phases  are  shown  in  Table  7. 

x  +  xn  279  +  48 

- =  -  =  .9849 

n  +  ng  282  +  50 

convert  to  A  =  ln(.9849)/-3  =  .005058 


Table  7.  Bayesian  Binomial  Model 


Test 

Number  of  Missions  x+x 
Success  Attempt  — — 

n+n 

(X)  (N) 

0  A 

0 

Confidence 

Lower 

Limit' 

Upper 

Phase 

I 

279 

282 

.  9849 

.005058 

.002465  . 

008082 

Phase 

II 

125 

126 

.9830 

.005731 

.002115  . 

01016 

Phase 

I 

261 

264 

.  9841 

.005351 

.002608  . 

008615 

Extension 

Combined 

665 

672 

.9875 

.004181 

.002525  . 

006038 

Bayesian  Binomial  Probability  Interval .  The  TBPI 
for  the  £(x+Xg,n+nQ)  posterior  distribution  given  by 


62 


1 


equation  (20)  was  calculated  for  Phase  I  data  as  shown 
below.  Other  phases  are  included  in  Table  7. 


Lower  (279+48) *F.95  (2*279+2*48,2*282+2*50-2*279-2*48) 

Limit  =  - 

282+50-279-48+ (27  9+48) *F.95  (2*279+2*48, 


2*282+2*50-2*279-2*48) 


327*F. 95(654, 10) 
5+327*F. 95(654, 10) 
to  convert  to  A,  ln(.9925)/-3  = 


327*2.06 

5+327*2.06 

.002465 


.9926 


Upper  279  +  48 

Limit  =  - 

279+48+ (282+50-279-48) *F.93 (2*282+2*50-2*279- 


2*48,  2*279+2*48) 

327  327 

327+5*F. 95(10, 654)  327+5*1.605 


.9761 


to  convert  to  ln(.9761)/-3  =  .008082 

Analysis .  These  Bayesian  models  had  point 
estimates  for  each  of  the  individual  test  phases  that  were 
consistently  higher  than  the  classical  estimates.  Those  for 
the  combined  values  were  much  closer,  indicating  the 
sensitivity  of  the  posterior  to  the  size  of  the  sample. 
Again,  all  of  these  point  estimates  were  significantly  lower 
than  the  flight  test  results.  In  contrast  to  the  classical 
values,  the  confidence  intervals  for  the  Bayesian 
exponential  approach  were  wider  for  each  of  the  four  test 
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cases  than  those  of  the  Bayesian  binomial  approach,  though 
not  by  an  appreciable  amount. 

Bayesian  Expert  Information  Model .  The  expert 
information  model  made  use  of  the  judgement  of  weapon  system 
experts  to  construct  Subjective  Probability  Distributions 
(SPD)  that  reflect  the  relative  uncertainty  of  their 
judgements.  These  SPD' s  were  then  modified  by  the  Bayesian 
process  to  incorporate  new  information  and  used  as 
management  tools  for  decision  making.  This  research  used  a 
modification  of  the  Kaplan  model  explained  in  Chapters  II  & 
III  to  obtain  a  point  estimate  and  probability  interval  for 
the  failure  rate  of  the  CMGS  based  on  the  steps  outlined  in 
the  methodology. 

Bayesian  Expert  Information  Prior.  The  Final 
Prior  was  the  product  of  a  series  of  iterative  updates  with 
each  successive  posterior  distribution  being  the  next  prior 
distribution.  The  calculations  for  each  update  are  provided 
in  Table  8,  and  a  sample  calculation  is  demonstrated  for  the 
generation  of  the  Second  Prior  from  the  First  Prior. 

Step  1.  The  specified  MTBF  for  the  CMGS  was  256  hours 
(29:1D),  which  corresponded  to  a  failure  rate  of  .003906. 
Using  a  three  hour  standard  mission,  this  failure  rate  had  a 
reliability  of  .9884.  In  addition  to  this  specified  value, 
there  were  predicted  values  for  the  CMGS  reliability  based 
on  parts  counts,  etc..  The  predicted  reliability  of  the 
RMUC  (9:3-70)  was  .9905,  with  a  failure  rate  of  .003182,  and 


it  may  be  regarded  as  an  upper  boundary  of  the  CMGS 
reliability  envelope.  The  predicted  CMGS  reliability  was 
.9868,  slightly  less  than  the  above  specified  value,  with  an 
associated  failure  rate  of  .004429.  It  was  considered 
unlikely  that  the  failure  rates  of  the  CMGS  would  deviate 
greatly  from  these  values.  In  order  to  account  for  the 
variance,  the  range  of  values  for  the  First  Prior  was 
expanded  to  .001  through  .006.  Given  the  range,  it  was 
necessary  to  attach  a  measure  of  the  uncertainty  inherent  in 
value. 

Expert  information  was  required  to  evaluate  the  risk  or 
uncertainty.  The  SPD  selected  to  represent  the  predicted 
reliability  and  associated  failure  rates  is  plotted  in 
Figure  9;  it  shows  higher  confidence  in  the  central  values 
of  the  range.  This  First  Prior  was  based  on  the  subjective 
determination  of  available  information  by  the  informed 
source . 

Step  2.  The  specified  MTBF  requirement  (35:6)  for  a 
similar  component,  the  ALCM  INE,  was  395  hours.  The 
demonstrated  MTBF  was  424  hours  in  the  period  from  30 
November  1981  through  31  December  1986,  involving 
approximately  576  failures  in  91,114  hours  for  1,687  ALCM 
INE  units  (see  Appendix  B) .  The  29  hour  increase 
represented  a  7.3418%  increase  over  the  specified  MTBF.  It 
should  be  noted  that  these  units  were  built  during  FY83  and 
FY84  production  lots,  the  same  production  lots  used  for  the 
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Table  8.  Bayesian  Expert- 
Information  First  through  Third  Priors 


First  Prior  =  SPD 

based  on 

CMGS  Specification 

MTBF  = 

256  hrs 

Ai  0.001 

0.002 

0.003 

0.004 

0.005 

0.006 

pl(Ai)  0.1 

0.15- 

0.25 

0.25 

0.15 

0.1 

Second  Prior  =  First  Prior 

undated 

with  ALCM 

INE  6. 

842% 

decrease . 

Ai  0.001 

0.002 

0.003 

0.004 

0.005 

0.006 

pl(Ai)  0.1 

0.15 

0.25 

0.25 

0.15 

0.1 

R=exp(-3  )  = 

0.9970 

0.9940 

0.9910 

0.9880 

0.9851 

0.9821 

R+. 0007925  = 

0.9977 

0.9948 

0.9918 

0.9888 

0.9859 

0.9829 

A(adj.)  7.35E-4  0 

.001734  0 

.002734  0 

.003733  0 

.004732 

0.00573 

(.3/. 7)  0.145 

0.18 

0.25 

0.22 

0.135 

0.07 

pi  ( Ai)  p  ( E  |  Ai;  = 

0.0145 

0.027 

0.0625 

0.055 

0.02025 

0.007 

/£  (p  ( £  1  Ai)  = 

p2 ( Ai) =  0.07785 

0.1449 

0.3355 

0.2953 

0.1087 

0.03758 

Third  Prior  =  Second  Prior 

updated  with  FCII 

reliability 

decrease  .005205 

R=exp  (-3  ) -.005205 

0.9917 

0.9888 

0.9858 

0.9828 

0.9799 

0.9769 

Ai  0.002745  0 

.00375  0 

.004755  0 

.005761  0 

.006766 

0.00777 

(.25/. 75)  0.01946 

0.09463 

0.1926 

0.3255 

0.2486 

0.1191 

Ai(adj)  0.001 

0.002 

0.003 

0.004 

0.005 

0.006 

p2(Ai)  0.07785 

0.1449 

0.3355 

0.2953 

0.1087 

0.0375 

p(E|Xi)  0.01946 

0.09463 

0.1926 

0.3255 

0.2486 

0.1191 

p2 (Ai) *p  (  E | Ai)  = 

0.001515 

0.01371 

0.06463 

0.09612 

0.02703 

0.00447 

/2 (p  (E  1  Ai) 

p3 ( Ai) =  0.007302 

0.06611 

0.3114 

0.4632 

0.1302 

0.0215 

flight  database  calculations  on  the  CMGS  units.  The  MTBF 
increase  corresponded  to  a  6.8414  %  decrease  in  the  failure 
rate  A.  This,  in  turn,  resulted' in  an  increase  in  the 
reliability  of  the  CMGS  units,  calculated  over  a  standard 
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three  hour  mission.  There  was  a  shift  down  in  the  failure 
rate  of  approximately  .0003  for  each  .001  division  of  A. 

For  example,  the  previous  probability  density  for  A  = 
.006  was  .10.  With  the  shift,  A  =  .005731,  leaving  70  %  of 
the  original  probability  density  with  the  .006  category,  and 
transferring  30%  to  the  .005  category.  This  continued  to 
the  .001  category,  which  kept  its  .10  and  the  additional  30% 
from  category  .002  for  a  grand  total  of  .1450.  Using  Bayes' 
Theorem,  it  was  straightforward  to  perform  the  indicated 
mathematical  operations  and  normalize  the  function  so  the 
area  under  the  SPD  was  equal  to  1.0.  The  resulting  SPD  is 
the  Second  Prior. 

The  Second  Prior  had  greater  probability  density 
concentrated  in  the  central  values  of  the  A  range.  Thus, 
the  effect  of  the  ALCM  INE  data  was  to  both  reinforce  the 
original  central  tendency  and  shift  it  toward  lower  values. 

Step  3.  The  design  weaknesses  identified  in  the  Ground 
Launched  Cruise  Missile  Flight  Critical  Item  Investigation 
(11:2.1-4.16)  were  used  to  update  the  Second  Prior.  Table  9 
tabulates  the  weaknesses  and  their  effect  on  the  CMGS 
reliability.  These  values  reduced  the  reliability  estimates 
and  increase  the  probability  of  observing  higher  failure 
rates.  The  SPD  for  A  was  therefore  adjusted  by  shifting  it 
toward  higher  failure  rates  by  an  amount  corresponding  to 
the  reduced  reliability,  .0052050.  This  factor  was  the 
summation  of  risk  values  tied  to  critical  failure  modes 
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Table  9.  GLCM  WSAP  Flight  Critical  Item  Design  Weaknesses 


Failure 

Modes 

(FCI's) 

Component 

Failure 

Probability 

Reliability 

Impact 

41,223 

(2824) 

Missile 

— 

— 

36,272 

(2280) 

CMGS 

- 

- 

33,363 

(2157) 

RMUC 

- 

- 

Not  Tested 

855 

II 

>  1 

X 

10-6 

8.55  x  10-4 

II 

31 

Other 

>  5 

X 

10-5 

1.55  x  10-3 

It 

28 

CMGS 

>  1 

X 

10-4 

2.8  x  10-3 

Total 

1 _ 

916 

5 

.205  x  10-3 

that  were  not  tested  adequately.  "Inadequately  tested"  was 
a  term  defined  by  the  GLCM  WSAP  FCII  based  on  the  level  of 
testing  performed  on  each  component,  the  criticality  of  the 
failure  modes  and  probability  of  failure. 

The  decreased  reliability  of  .005205  vras  subtracted 
from  the  calculated  reliability  values  for  each  and  a  new 
Xj  was  obtained  for  the  resulting  decreased  reliability. 

For  example,  the  new  Aj  for  A^  =  .001  was  In  (.997004- 
.005205) /-3,  which  equaled  .002745.  Because  this  value  was 
about  75%  across  the  interval  between  .002  and  .003,  the 
curve  was  shifted  to  the  right  by  the  same  75  %.  Thus,  25  % 
of  the  probability  density  for  .001  remained  at  .001.  The 
remaining  75%  of  .077852  +  25%  of  .144966,  the  probability 
density  for  .002,  made  up  the  new  probability  density  for 
.002,  or  .094607. 

These  new  probability  values  were  used  to  update  the 
Second  Prior  for  the  design  weaknesses  —  the  resulting 
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posterior  was  the  Third  Prior.  As  in  Step  2  above,  the 
central  tendency  was  reinforced,  but  with  a  shift  toward 
higher  values.  Although  the  ALCM  INE  and  FCII  updates  might 
have  reasonably  been  expected  to  cancel  their  respective 
effects,  there  was  a  new  combined  probability  density  of 
77.5%  for  .003  £  A  ^  .004  versus  the  First  Prior 
probability  density  of  50%  for  the  same  values. 

Step  4.  The  Environmental  Stress  Screening  and 
acceptance  testing  results  (32-34)  were  not  suitable  for 
inclusion  in  this  analysis.  Examination  of  the  data 
indicated  that  the  CMGS  component  screening  process  was 
allowing  an  unacceptable  number  of  quality  escapes  and 
infant  mortality  to  pass  on  to  the  missile  acceptance 
testing.  Additionally,  the  missile  Acceptance  Test 
Procedure  had  ESS  and  functional  checkouts  intermingled  to 
an  extent  that  it  was  not  possible  to  extract  meaningful 
data  for  either.  Finally,  data  was  not  available  on  the 
number  of  attempts  necessary  to  completely  pass  through  all 
the  tests.  Although  there  was  first  pass  yield  data 
available,  because  of  the  problems  with  test  structure  and 
screening  levels  referenced  above,  this  data  was  not 
suitable  for  the  research  (see  Appendix  C) .  This  step  was 
bypassed. 

Step  5a.  (Exponential)  —  Flight  test  data  was  used  to 
update  the  Third  Prior  and  form  the  posterior  that  was  the 
Final  Prior.  For  the  exponential  approach,  equation  (21) 
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was  used  with  n  failures  in  time  T.  Considering  flight  test 
data  (26)  for  CMGS  units  produced  in  FY83  and  later, 
n  =  2  and  T  =  75.2.  Again  using  the  standard  three  hour 
mission,  it  was  possible  to  calculate  the  probability, 
p(E|Ai),  of  two  failures  in  75.2  hours,  given  each  of  the 
failure  rates,  Ai.  in  the  Third  Prior,  for  1  -  .001, 
p(E|Al)  =  .002623,  and  the  corresponding  Final  Prior  value 
was  .00063.  Other  values  are  tabulated  in  Table  10. 

The  tinal  Prior  was  ready  to  use  for  calculating 
exponential  posterior  distributions  for  each  of  the  phases 
->f  the  CMGS  Reliability  Demonstration.  Before  proceeding, 
the  Final  Prior  needed  to  be  determined  for  the  binomial 
approach. 

Step  5b.  (Binomial)  —  The  Final  Prior  was  calculated 
by  updating  the  Third  Prior  with  the  results  of  flight 
testing  of  the  current  configuration,  48  successful  out  of 

Table  10.  Bayesian  Expert  Information 
Exponential  Final  Prior 

/'  '■  - "  — - — — — —  — ■■■ 

Final  Prior  =  Third  prior  updated  by  Flight  Test  times 
for  FY83  and  on.  2  failures,  75.2  hours. 

Ai  0.001  0.002  0.003  0.004  0.005  0.006 

p3  (Ai)  0.007302  0.06611  0.3114  0.4632  0.1302  0.02157 

p(E|Xi)  0.002623  0.009731  0.02031  0.03349  0.04854  0.06483 

P3(Ai)  *p  (  E  |  Ai)  = 

1.90E-05  0.000643  0.006326  0.01551  0.006323  0.00139 

/2<p(ei  *i> 

pF  (Ai) =  0.00063  0.02129  0.2093  0.5133  0.2092  0.04628 
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50  attempts.  Equation  (21)  was  used  in  the  binomial  format 
and  the  calculations  were  made  in  a  similar  fashion  as  in 
the  exponential  approach  once  P(E|Ai)  had  been  determined. 
For  1  =  .001,  P(ElXi)  =  .000867  and  the  resulting  posterior 
probability  was  .002257.  Other  ralues  are  tabulated  in 
Table  11. 

Table  11.  Bayesian  Expert  Information  Binomial  Final  Prior 


Final 

Prior  =  Third  Prior  updated  by  Flight  Test 

results 

using  FY83  &  on, 

48/50 

Ai 

0.001 

0.002 

0.003  0.004 

0.005 

0.006 

p3  (Ai) 

0.007302 

0.06611 

0.3114  0.4632 

0.1302 

0.0215 

P(E|  Xi) 

0.001168 

0.004451 

0.009544  0.01617 

0.02408 

0.0330 

p3  (Ai)  * 

p  (E  |  Ai)  = 

8.53E-06 

0.00029  0 

.002973  0.00749 

0.003137' 

0.00071 

\2(p  ( £  1  Ai) 

PF (Ai) = 

0.000583 

0.02013 

0.2034  0.5125 

0.2146 

0.0487 

.  ,  ■  J 

Both  the  exponential  and  binomial  approaches  resulted 
in  similar  Final  Priors.  The  chief  impact  of  the  flight 
test  data  was  to  further  centralize  the  probability  density 
and  simultaneously  shift  it  toward  slightly  higher  values  of 

A  • 

Bayesian  Expert  Information  Posteriors .  The  Final 
Priors  were  used  with  the  results  of  the  CMGS  Reliability 
Demonstration  to  calculate  the  Posteriors  for  each  of  the 
three  test  phases  and  the  combination  of  all  three.  The 
calculations  were  based  on  equation  (21)  in  the  same  manner 


* 


\ 
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as  in  Steps  4. a  and  5.b  above.  Exponential  results  are 
tabulated  in  Table  12;  binomial  results  are  in  Table  13. 

Bayesian  Expert  Information  Probability  Intervals. 
The  TBPI  was  calculated  using  an  cK  value  of  .1, 
corresponding  to  a  90  %  confidence  limit.  The  total  area 
under  the  SPD  was  equal  to  one.  The  90  %  TBPI  was  defined 
by  the  two  endpoints  of  the  shortest  interval  that 
encompassed  90%  of  the  area  under  the  distribution.  To 
calculate  the  interval,  ten  increments  were  spaced  between 
each  A  value  and  the  area  was  set  equal  to  .90.  An 
estimate  of  the  interval  length  was  made  and  moved  along  the 
A  x-axis  incrementally.  The  shortest  interval  that  still 
encompassed  an  area  of  .90  was  the  TBPI.  The  point 
estimates  and  TBPI  endpoints  are  shown  in  Table  14. 

Analysis .  The  expert  information  exponential  and 
binomial  approaches  were  remarkably  consistent.  The  point 
estimates  were  in  the  range  of  values  for  the  other  models, 
and  were  significantly  lower  than  the  flight  test  results. 
The  real  difference  in  this  approach  was  the  noticeably 
narrower  confidence  intervals.  In  every  case,  the  width  did 
not  exceed  .003,  less  than  any  other  interval  calculated  in 
this  research. 

A  summary  of  the  results  will  be  presented  in  Chapter 
V.  The  potential  applications  of  this  research  will  be 
discussed,  along  with  suggestions  for  future  investigations. 
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Table  12.  Bayesian  Expert  Information 
Exponential  Posteriors 


j  First  Case  -  Exponential  Phase  I 

946.486  hours,  3 

failures 

A  i 

0.001  0.002 

0.003 

0.004 

0.005 

0.006 

pF  (Ai) 

0.00063  0.02129 

0.2093 

0.5133 

0.2092 

0.0462 

pUlAi) 

0.05485  0.1703 

0.2230 

0.2052 

0.1555 

0.1043 

pF (Ai) * 

P  (  E  1  Ai)  /S (p  (  E  1  Ai )  = 

0.00018  0.01878 

0.2419 

0.5456 

0.1686 

0.02501 

Point  Estimate  = 

0.003939 

Second 

Case  -  Exponential 

P-hase  II 

,  423.11 

hours,  1 

failure 

Ai 

0.001  0.002 

0.003 

0.004 

0.005 

0.006 

pF (Ai) 

0.00063  0.02129 

0.2093 

0.5133 

0.2092 

0.04628 

P(EIU) 

0.2771  0.3631 

0.3567 

0.3115 

0.2551 

0.2005 

pF (Ai) * 

p(E|Ai)/2(p(£|Ai)  = 

0.000576  0.02533 

0.2447 

0.5241 

0.1749 

0.03041 

Point  Estimate  = 

0.003939 

Third  Case  -  Exponential  Phase  I  Extension, 

881.4128  hours,  i 

3  failures 

Ai 

0.001  0.002 

0.003 

0.004 

0.005 

0.006 

PF (Ai) 

0.00063  0.02129 

0.2093 

0.5133 

0.2092 

0.04628 

p(EIAi) 

0.04727  0.1566 

0.2189 

0.2149 

0.1739 

0.1244 

pF (Ai) * 

P (E 1 Ai) /£ (p (p | Ai) = 

0.000149  0.01653 

0.2272 

0.5471 

0.1804 

0.02856 

Point  Estimate  = 

0.003977 

Fourth 

Case  -  Exponential 

Combined  2251  hours,  7  failures 

Ai 

0.001  0.002 

0,003 

0.004 

0.005 

0.006 

pF  (Ai) 

0.00063  0.02129 

T.2093 

0.5133 

0.2092 

0.0462& 

p(ElAi) 

0.006118  0.08245 

J .  1483 

0.1170 

G. 05874 

0.02216 

pF(Ai)  * 

P(E  1  Ai)  /£(P  (  fcl  Ai)  = 

3.65E-05  0.01653 

0.2924 

0.5656 

0.1158 

0.00966 

Point  Estimate 

=  0.00381 

/tysuao  /tyjqoqoy 


Figure  7.  Bayesian  Expert  Information 
Exponential  Approach 


74 


Failure  Rate 


Table  13.  Bayesian  Expert  Information  Binomial  Posteriors 


First  Case  -  Binomial  Phase  I  279/282 

Ai  0.001  0.002 

0.003 

0.004 

0.005 

0.006 

pF  <  Ai )  0.000583  0.02013 

0.2034 

0.5125 

0.2146 

0.04377 

P  (E 1 Ai)  0.002797  0.01692 

0.04317 

0.07735 

0.1141 

0.1490 

pF(M)  *p(£  |  Ai)  /£(p(£|M)  = 

2.03E-05  0.00423 

0.1091 

0.4922 

0.3042 

0.09023 

Point  Estimate  = 

0.004367 

Second  Case  -  Binomial 

Phase  II 

125/126 

Ai  0.001  0.002 

0.003 

0.004 

0.005 

0.006 

pF (Ai)  0.000583  0.02013 

0.2034 

0.5125 

0.2146 

0.04877 

p  (E 1 Ai)  0.1111  0.1962 

0.2596 

0.3053 

0.3366 

0.3563 

pF(M)*p(E|Xi)/fi(p(E|Ai)  = 

0.000214  0.01304 

0.1743 

0.5166 

0.2385 

0.0573 

Point  Estimate  = 

0.004152 

Third  Case  -  Binomial  Phase  I  extension 

261/264 

Ai  0.001  0.002 

0.003 

0.004 

0.005 

0.006 

pF(VL)  0.000583  0.02013 

0.2034 

0.5125 

0.2146 

0.04877 

p ( E 1 Ai)  0.002335  0.01438 

0.03736 

0.06816 

0.1024 

0.1361 

pF(Ai)*p(EIXi)/£(p(ElXi)  = 

1.91E-05  0.004053 

0.1064 

0.4889 

0.3077 

0.09293 

Point  Estimate  = 

0.004379 

Fourth  Case  -  Binomial 

Combined 

665/672 

Ai  0.001  0.002 

0.003 

0.004 

0.005 

0.006 

pF(Xi)  0.000583  0.02013 

0.2034 

0.5125 

0.2146 

0.04877 

p(E|Xi)  6.12E-06  0.000402 

0.003529 

0.01356 

0.03316 

0.0608 

pF(M)*p(£|M)/£(p(E|M)  = 

2.01E-07  0.000456 

0.04041 

0.3913 

0.4007 

0.1672 

Point  Estimate  0 

.004694 

Ajisibq  A^qoqay 


Figure  8.  Bayesian  Expert  Information  Binomial  Approach 


1st  Prior 


Taole  14.  Bayesian  Expert  Information  Probability  Intervals 


Test  Phase 

Point 

Estimate 

Probability 
Lower  Upper 

Interval 

Confidence 

Exponential  Approach: 

Phase  I 

.003946 

.0025 

.0051 

.9055 

Phase  II 

.003947 

.0025 

.0052 

.9087 

Phase  I  Extension 

.003984 

.0025 

.0051 

.  9006 

Combined  Phases 

.003815 

.0025 

.0048 

.9043 

Binomial  Approach: 

Phase  I 

.004372 

.0025 

.0054 

.9055 

Phase  II 

.004157 

.0026 

.0054 

.9116 

Phase  I  Extension 

.004384 

.0025 

.0054 

.9030 

Combined  Phases 

.004698 

.0035 

.0061 

.9090 
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V.  Recommendations 


Data  Analysis 

A  summary  of  the  point  estimates  for  A  and  the  width  of 
90%  confidence  intervals  is  displayed  in  Table  15  for  each 
of  the  models  and  CMGS  test  phases.  The  military  manager 
who  is  facing  a  weapon  systems  decision  will  often  request 
reliability  analyses  be  performed  to  determine  the  most 
likely  reliability  point  estimate.  It  is  often  difficult  to 
make  the  decision  without  knowing  something  about  the 
uncertainty  or  risk  associated  with  accepting  that  point 
estimate.  The  savvy  military  manager  will  also  request  an 
analysis  be  performed  to  determine  the  confidence  interval 
surrounding  the  point  estimate.  As  was  explained  earlier  in 
Chapter  2,  the  classical  confidence  interval  is  stating  that 
for  a  certain  percentage  of  trials,  the  true  population 
value  will  fall  within  a  range  about  the  point  estimate. 

The  Total  Bayesian  Probability  Interval  is  stating  the 
probability  that  the  true  value  is  within  a  range  about  the 
point  estimate  for  each  trial.  The  common  interpretation  of 
each  of  these  is  that  they  represent  the  risk  assumed  by  the 
manager  in  accepting  the  point  estimate  as  the  true  value. 

If  the  intervals  are  very  narrow,  the  manager  believes  that 
even  if  che  point  estimate  is  not  correct,  it  is  very  close 
to  the  actual  value.  Similarly,  if  the  interval  is  wide, 
the  true  value  could  take  on  a  range  of  values,  some  of 
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Table  15.  Point  Estimates  and  Confidence  Interval  Widths 


Phase  I 

Classic  Expon 

.003170 

.005204 

Phase  I 

Classic  Binomial 

.003565 

.006583 

Phase  I 

Bayes  Expon 

.004894 

.007033 

Phase  I 

Bayes  Binomial 

.005058 

.005617 

Phase  I 

Expert  Expon 

.003939 

.0026 

Phase  I 

Expert  Binomial 

.004367 

.0029 

Phase  II 

Classic  Expon 

.002364 

.0062401 

Phase  II 

Classic  Binomial 

.002656 

.0100613 

Phase  II 

Bayes  Expon 

.006020 

.01099 

Phase  II 

Bayes  Binomial 

.005731 

.008045 

Phase  II 

Expert  Expon 

.003939 

.0027 

Phase  II 

Expert  Binomial 

.004152 

.0028 

Phase  I  Ex 

Classic  Expon 

.003404 

.005592 

Phase  I  Ex 

Classic  Binomial 

.003810 

.007024 

Phase  I  Ex 

Bayes  Expon 

.005338 

.007669 

Phase  I  Ex 

Bayes  Binomial 

.005351 

.006007 

Phase  I  Ex 

Expert  Expon 

.003977 

.0026 

Phase  I  Ex 

Expert  Binomial 

.004379 

.0029 

Combined 

Classic  Expon 

.003110 

.003491 

Combined 

Classic  Binomial 

.003490 

.003981 

Combined 

Bayes  Expon 

.003869 

. 004187 

Combined 

Bayes  Binomial 

.004181 

.003513 

Combined 

Expert  Expon 

.003810 

.0023 

Combined 

Expert  Binomial 

.004694 

.0026 
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which  might  be  quite  different  from  the  point  estimate. 
Therefore,  for  the  purposes  of  managerial  decision-making, 
the  shorter  the  confidence  interval  surrounding  the  point 
estimate,  the  lower  the  risk  in  accepting  the  point 
estimate.  The  discussion  of  these  results  and  a  comparison 
with  the  flight  test  point  values  follows. 

Models .  Within  each  of  the  different  model  approaches, 
relatively  consistent  point  estimates  for  A  were  obtained 
for  each  of  the  test  phases  and  combined  phases  data.  For 
example,  Table  15  shows  the  classical  exponential  point 
estimates  for  Phase  I,  II,  I-extension,  and  combined  phases 
data  are  .003170,  .002364,  .003404,  and  .003110  . 

There  is  a  similar  trend  among  the  confidence  interval 
widths  through  the  three  test  phases,  .005204,  .006240,  and 
.005592  for  the  classical  exporontial  model,  with  relatively 
consistent  widths  for  each  model  approach.  The  confidence 
interval  widths  for  the  combined  phases  data,  .003491  for 
the  classical  exponential  model,  arc  shorter  for  all  the 
approaches,  predictably  the  result  given  the  greater  amount 
of  data  used  in  those  calculations. 

The  point  estimates  for  the  classical  approaches  were 
always  the  smallest  values,  followed  by  the  expert 
information  and  then  Bayesian  approaches.  This  reflects  the 
relative  effect  of  flight  test  data  in  the  prior 
distributions.  For  example,  the  classical  approach  only 
considers  the  CMGS  Reliability  Demonstration  data;  the 
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flight  test  data  has  no  effect.  Hence,  this  approach  would 
show  the  greatest  variance  from  the  flight  test  point 
estimates . 

The  expert  information  approach  confidence  interval 
widths  were  the  smallest  in  every  case.  The  classical 
approach  intervals  were  slightly  smaller  than  those  of  the 
Bayesian  approach,  but  still  large  compared  to  the  expert 
information  values. 

Flight  Test  Results .  The  CMGS  flight  test  failures 
were  concluded  to  be  distributed  exponentially  when  examined 
over  the  total  flight  test  program.  It  was  not  possible  to 
conduct  a  meaningful  Goodness-of-Fit  test  for  the  flight 
test  failures  of  CMGS  units  with  comparable  configurations 
to  those  used  in  the  CMGS  Reliability  Demonstration  because 
of  the  limited  number  of  failures.  A  Weibull  plot  of  the 
total  failures  shown  in  Figure  9  indicates  that  the  shape 
parameter was  approximately  2.0,  significantly  above  the 
0.0  to  1.0  range  associated  with  a  strictly  exponential 
failure  rate.  This  is  also  supported  by  the  differences 
between  exponential  and  binomial  approaches  for  each  of  the 
models.  It  appears  that  there  may  be  other  failure 
mechanisms  at  work  based  on  the  limited  data  available. 

Because  the  CMGS  configuration  changes  that  were 
intended  to  improve  reliability  were  introduced  over  a 
period  of  time,  it  would  be  reasonable  to  expect  a  range  of 
reliability  characteristics  corresponding  to  the  different 
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CUMULATIVE  PER  CENT  FAILURE 
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Figure  9,  Flight  Test  Failure  Weibull  Flot 
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configurations.  However,  because  the  changes  were  all 
originally  classified  as  producitility  improvements  that 
should  not  appreciably  change  the  form,  fit,  or  function  of 
the  CMGS  units,  it  is  difficult  to  isolate  specific 
reliability  improvements  to  each  change.  In  addition,  the 
contractor  had  no  obligation  to  include  the  changes  in  every 
unit  that  was  produced. 

Based  on  the  differences  in  failure  rates  calculated 
using  the  different  samples  of  the  flight  test  database,  it 
seems  likely  that  the  configuration  changes  did  indeed 
affect  the  reliability  of  the  CMGS.  For  example,  the 
failure  rate  for  the  FY84  units  is  .01830  vice  .0680  for  the 
total  flight  test  program  using  the  exponential  approach, 
and  .008033  vice  .03543  using  the  binomial  approach. 

Although  every  effort  was  made  in  designing  the  CMGS 
Reliability  Demonstration  to  simulate  the  operational 
environments  experienced  by  the  CMGS,  the  differences 
between  the  Bayesian  and  classical  approaches  indicate  some 
portions  of  the  test  environment  were  more  benign  than  found 
in  the  real  world.  Because  che  point  estimates  from  the 
Bayesian  approaches  are  closer  to  the  flight  test  results 
than  the  classical  statistical  inference  approaches,  the 
Bayesian  approaches  are  better  tools  for  evaluating  the 
results  of  the  CMGS  Reliability  Demonstration. 
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Other  Applications 


The  final  step  in  this  research  is  the  recommendation 
of  further  research  in  the  area  of  Bayesian  reliability. 
Three  proposed  applications  are  described  in  the  next 
sections . 

Advanced  Cruise  Missile .  A  clear  application  of  this 
technology  is  the  generalization  of  the  findings  to  the  next 
generation  cruise  missiles.  The  Advanced  Cruise  Missile 
(ACM)  Navigation  and  Guidance  System  has  the  same 
approximate  degree  of  complexity  as  the  CMGS  and  performs  an 
equivalent  function.  The  Bayesian  expert  information  model 
should  be  developed  for  the  ACM  and  used  to  build  prior 
distributions  upon  which  to  base  reliability  predictions. 
Because  the  ACM  is  projected  to  have  a  higher  designed 
reliability,  the  disadvantages  in  conducting  testing  in 
support  of  classical  statistical  inference  analyses  would  be 
expected  to  be  greater  as  well. 

Avionics  Integrity  Program  (AVIP) .  One  of  the  chief 
thrusts  of  the  ASD  AVIP  program  is  to  change  the  Air  Force's 
perception  of  reliability  from  a  reliance  on  MTBF  to  an 
emphasis  on  the  "Failure  Free  Operational  Period".  Of 
course,  with  a  constant  failure  rate  model,  such  as  the 
exponential  distribution,  there  is  always  a  finite 
probability  of  failures.  If  the  hardware  can  be  designed  in 
so  as  to  locate  the  requ.  *ed  operating  period  within  an 

84 


7 


interval  of  relatively  low  probability  of  failure,  such  as 
in  the  tail  portion  of  a  failure  distribution,  the  desired 
result  can  be  obtained. 

The  use  of  Bayesian  methodology,  particularly  the 
expert  information  approach,  can  specify  shorter  confidence 
intervals  for  a  given  amount  of  test  data,  resulting  in  a 
lower  probability  of  failure  for  the  tails  of  the 
distribution.  By  specifying  the  required  operating  period 
outside  of  the  confidence  interval,  the  very  low 
probabilities  of  failure  can  be  achieved  (see  Figure  10) . 


- - - - — - - 

Figure  10.  AVIP  Failure  Free  Operational  Period  (FFOP) 

Foreign  Technology  Evaluation .  The  Bayesian 
approaches,  in  particular  the  expert  information  approaches, 
seem  to  be  good  candidates  for  evaluating  the  reliability  of 
foreign  technology.  In  many  of  these ’ scenarios,  the  analyst 
has  some  knowledge  of  the  current  threat  or  capability  and 
considerably  more  information  regarding  earlier  types  or 
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versions.  The  Bayesian  approach  allows  the  analyst  to 
assess  the  validity  of  new  information  and  weight  the 
reliability  assessment  accordingly.  As  more  information  is 
received,  the  prior  distributions  can  be  easily  updated, 
further  refining  the  analysis. 

Summary 

The  Bayesian  models  defined  in  this  research  are 
capable  of  specifying  shorter  confidence  intervals  about 
point  estimates  for  cruise  missile  electronic  component 
failure  raters  than  the  traditional  classical  statistical  ' 
inference  models.  The  expert  information  model  is 
particularly  useful  in  that  it  can  easily  utilize  a  wide 
variety  of  data  sources,  enabling  more  accurate  point 
estimation  and  confidence  interval  calculation.  Its  point 
estimates  were  more  representative  of  the  flight  test 
results,  and  thus  presumably  the  true  failure  rate,  than 
were  the  classical  models.  The  confidence  intervals  were 
clearly  shorter  than  for  any  of  the  classical  or  Bayesian 
models,  making  them  particularly  useful  for  managerial 
decision-making.  Because  there  is  uncertainty  in  verifying 
the  constancy  of  the  CMGS  flight  test  failure  rate,  the 
expert  information  binomial  model  is  recommended  for  future 
work  with  the  CMGS  units. 
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Appendix  A:  TLAM  Flight  Test  Data  (26) 


Date  Missile 

Flight  Time  MSL 

Success  CMCS 

Succe33  CM* 

790717  AL2 

1 

2.33  5 

F 

79 

790001  AL1 

1 

2  .  A  A  5 

S 

79 

790900  AL3 

1 

0.02  F 

N 

79 

790929  ALA 

1 

A  .  0  A  5 

r* 

J 

79 

791027  AL2 

O 

2. 

3. 9  A  5 

/• 

79 

791115  AL6 

1 

0.02  F 

N 

79 

791206  AL1 

2 

0.85  F 

c* 

•J 

7  9 

791227  ALA 

2 

0 . 7  A  F 

s 

79 

80012A  AL7 

1 

3.90  5 

5 

7  9 

800208  AL5 

1 

A  .  1 0  Z 

r> 

79 

000516  T16 

1 

1.76  S 

s 

79 

800816  T15 

1 

0.01  F 

N 

79 

801126  T16 

2 

1.97  5 

F 

79 

010215  T17 

3 

1.A9. 5 

s 

79 

810328  T50 

1 

1  .  AA  S 

O 

79 

810710  T51 

0.99  S 

s 

79 

810730  T50 

2 

0.98  F 

F 

79 

810919  T17 

A 

1.35  S 

r» 

-J 

79 

811027  T52 

0.99  5 

5 

79 

811107  T5 A 

0.01  F 

N 

7  9 

8 1 1 21 A  T53 

0.99  F 

S 

79 

820225  T72 

1 

1.90  S 

F 

7  9 

820325  T73 

1 

1.89  S 

F 

79 

820330  T56 

0.01  F 

N 

7  9 

020519  T7A 

1 

1 .96  5 

r 

ki 

*7 

4  7 

820521  T55 
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Appendix  D:  Tomahawk  Burn-In  Test  Report  Excerpts  (33:  2-10) 


TLAMGS  YIELDS 
LEVEL  1  VS.  LEVEL  2 


OCT  -  DEC  87  JAN  •  MAR  88  APRIL  -  JUN  88  JUL  -  SEPT 


COMPLETED 


34 

11 

25 

6 

3 

1 

82% 

73% 

98% 

OS  LEVEL  (LEVEL  2) 
MISSILE  LEVEL  (LEVEL  1} 


FIGURE  2-4 


Figure  2-6  above  shows  TLAM  Guidance  Set  yields  for  Level  1  and  Level  2 
testing  on  a  quarterly  basis  and  for  this  reporting  period  (July  -  September  1988) 
for  comparison.  This  information  can  be  used  to  discern  two  things:  first, 
whether  yield  at  Level  2  testing  is  improving  and  second,  whether  Levell 
(missile  level)  yield  is  less  or  greater  than  Level  2  and  if  that  gap  is  changing. 

Ground  rules  for  data  in  this  figure  are  as  follows: 

-  Only  first  attempt  at  passing  Level  2  ATP  and  first  Level  1  Guidance  Set 
failures  considered. 

-  Only  units  completing  testing  during  each  period  considered. 

-  Only  proven  or  pending  hardware  failures  included. 


Level  2  yield  is  down  for  the  3rd  quarter.  The  major  discrepancy  is  against 
RMUCs  with  four  failures.  These  items  have  been  tested  on  Litton's  TATE 
and  are  being  dispositioned  for  return  to  the  vendor.  Level  t  yield  remains 
essentially  the  same  this  quarter. 


Appendix  E:  Tomahawk  Burn-In  Test  Report  Excerpts  (34:  2-10) 


TLAMGS  YIELDS 
LEVEL  1  VS.  LEVEL  2 


JAN  *  MAR  ft  APR  •  JUN  17  JU.  -  SEPT  17  OCT -NOV  17 


1£VBJ  OS  MSL  GS  M5L  GSMSL  GSMSL 

FIGURE  2-* 


Figure  2-6  above  shows  TLAM  Guidance  Set  yields  (or  Level  1  and  Level  2 
testing  on  a  quarterly  basis  and  lor  this  reporting  period  (October  •  November) 
fcr  comparison.  This  information  can  be  used  to  discern  two  things:  first, 
whether  yield  at  Level  2  testing  is  improving  and  second,  whether  Levell  (missile 
level)  yield  is  less  or  greater  than  Level  2  and  if  that  gap  is  changing. 

Ground  rules  (or  data  in  this  figure  are  as  follows: 

-  Only  first  attempt  at  passing  Level  2  ATP  and  first  Level  1  Guidance  Set 
failures  considered. 

•  Only  units  completing  testing  during  each  period  considered. 

•  Only  proven  or  pending  hardware  failures  included. 

The  figure  shows  Level  1  yield  decreasing  while  Level  2  yield  is  improving  to 
100%.  This  disparity  between  guidance  set  level  testing  and  missile  level 
guidance  removals  is  not  desirable  and  indicates  that  Level  2  test  escapes 
(latent  detects)  are  likely  being  passed  along  to  missile  level.  Accordingly, 
the  positive  trend  at  Level  2  must  be  tampered  with  this  concern.  Further 
Level  1  discussion  is  found  back  in  section  1  under  figure  1-5,  while  Level  2 
detail  is  on  the  following  pages. 


Appendix  F:  Calculations 


Flight  Test  Classical  Exponential 
All  Data 


Lower  15.38 

Limit  - -  =  .04358 

2  *  176.44 

Upper  36.42 

Limit  - -  =  .10321 

2  *  176.44 


FY84 


Lower  .7107 

Limit  = -  =  .006502 

2  *  54.65 

Upper  5.991 

Limit  = -  =  .05481 

2  *  54.65 


95 


Flight  Test  Classical  Binomial 


All  Data 


Lower  (107+1)  *F  Qt- (2*107+2,  2*119-2*107) 

Limit  - - - - 

119-107+  (107  +  1) *F  g5  (2*107+2,  2*119-2*107) 

108*F  95  (216, 24)  108*1.555 


12+108*F  95  (216, 24) 


12+108*1.555 


and  converting  to  a  failure  rate,  P(t)  =  exp(-A*3), 
=  In  ( . 9333) /-3  =  .02301 


Upper 

Limit 


107+ (119-107+1) *F  95  (2*119-2*107+2,  2*107) 


107  +  13*F  95  (26, 214)  107+13*1.405 

=  In (R) / (-3)  =  In ( . 8542) /-3  =  .05253 


FY84 


8542 


Lower 

Limit 


(41  +  1) *F  95  (2*41+2,  2* 42-2*41) 

42-41+ (41+1) *F  95  (2*41+2,  2*42-2*41) 


42*F_95(84,2) 


42*10.1 


1+42*F<95(84,2) 


1+42*10.1 


=  In  (.  997  6) /-3  -  .0007849 


Upper 
Limit  = 


41+ (42-41  +  1) *F  QC  (2*42-2*41+2,  2*41) 

•  yJ 


41+2*F>95(4,82) 


41+2*2.01 


=  In (R) / (-3)  =  In (. 9107) /-3  =  .03118 


.9107 


96 


Classical  Exponential 


Phase  II 


Lower 

.1026 

Limit 

2  *  423.11 

Upper 

9.488 

Limit 

2  *  881.413 

Phase  I  Extension 

Lower 

2.733 

Limit 

2  *  881.413 

Upper 

12.592 

Limit 

2  *  881.413 

Combined  Phases 

Lower 

7.962 

Limit 

2  *  2251 

Upper 

23.68 

Limit 

2  *  2251 

.0001212 

.01121 

.001550 

.007142 

.001769 

.005260 


Classical  Binomial 
Pnase  II 


Lower  (125+1) *F  q,  (2*125+2,  2*126-2*125) 

Limit  = - --- - 

126-125+ (125+1) *F  Qc  (2*125+2,  2*126-2*125) 

126*F  95  (252 ,  2)  126*10.2 

1  +  126*F  95  (252,  2)  1  +  126*10.2 

and  converting  to  a  failure  rate,  R(t)  =  exp(-A*3),- 

=  In ( . 9992) /-3  =  .0002593 


Upper  125 

Limit  =  - 

125+ (126-125+1) *F  95  (2*126-2*125+2,  2*125) 

125  125 

(125+2*F  95  (4, 250)  125+2*1.96 

=  In (R) / (-3)  =  In  ( . 9696) /-3  =  .01029 


Phase  _I  Extension 

Lower  (261  +  1) *F  Qc  (2*261+2,  2*264-2*261) 

Limit  =  - - - 

264-261+ (261  +  1) *F  95  (2*261+2,  2*264-2*261) 

264*F  95(524,  6)  264*2.72 

1  +  126*F  95  (524 ,  6)  1+264*2.72 


=  ln(. 9958)/-3  =  .00140 


Upper  261 

Limit  =  - 

261+ (264-261  +  1) *F  95  (2*264-2*261+2,  2*261) 


98 


261 


261 


.9749 


261+4*F  95(8,522)  261+4*1.68 

=  In (R) / (-3)  =  In  ( . 9749) /-3  =  .008474 
Combined  Phases 


Lower  (665+1)  *F  p[- (2*665+2,  2*672-2*665) 

Limit  = - --- - 

672-665+(665+l) *F  95(2*665+2,  2*672-2*665) 

666*F  95  (1332,  14)  666*1.80 

7+666*F  95 (1332, 14)  7+666*1.80 


=  In  ( . 9942) /-3  =  .001941 


Upper 

Limit 


665 

665+ (672-665+1) *F  g5  (2*672-2*665+2,  2*665) 

665  665 

- - -  -  =  .9822 

( 665+8*F  95  (14,  1330)  665+8*1.51 

=  In (R) / (-3)  =  In  ( . 9822) /-3  =  .006001 


♦ 


f 

! 

i 

! 


99 


s  +  <* 


Bayesian  Exponential 
Phase  II 

1  +  2 

-  =  .006020 

t  +  £  423.11  +  75.2 

Lower  X2  Ql.(2*l  +  2*2)  1.635 

Limit  =  - --- -  - - -  .001641 

2*  (423.11  +  75.2)  996.62 

Upper  X2  nc(2*l  +  2*2)  12.59 

Limit  =  -  = -  =  .01263 

2*  (423.11  +  75.2)  996.62 

Phase  I  Extension 
s  +  oC  3  +  2 

- =  - =  .005338 

t  +  0  861.413  +75.2 

Lower  X2  QC.(2*3  +  2*2)  3.945 

Limit  - - - -  =  - -  .002106 

2*  (861.413  +  75.2)  1873.23 

Upper  X2  nc;  (2*3  +  2*2)  18.31 

Limit  =  ------ -  - -  =  .009775 

2*  (861.413  +  75.2)  1873.23 

Combined  Phases 
s  +  7  +  2 

- =  -  =  .003869 

t  +  0  2251  +75.2 

Lower  X2  q,.  (2*7  +  2*2)  9.39 

Limit  =  - - -  = - =  .002018 

2*  (2251  +  75.2)  4652.4 

Upper  X2  nc-(2*7  +  2*2)  28.87 

Limit  =  - - -  - - -  .006205 

2*  (2251  +  75.2)  4652.4 
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Bayesian  Binomial 
Phase  II 

125  +  48 

-  =  .9830  ,  convert  to  A, 

126  +  50 

A  =  In  (. 9830) /-3  =  .005731 


Lower  (125+48) *F.95 (2*125+2*48,2*126+2*50-2*125-2*48) 

Limit  =  - 

126+50-125-48+ (125+4  8) *F.95  (2*125+2*48, 


2*126+2*50-2*125-2*48) 

173*F. 95(346, 6)  173*2.725 

- -  - -  =  .9937 

3+173*F. 95(346, 6)  3+173*2.72 

to  convert  to  A,  ln(.9937)/-3  =  .002115 


Upper  125  +48 

Limit  =  - 

125+48+ (126+50-125-48) *F. 95 (2*126+2*50-2*125- 


2*48,  2*125+2*48) 

173  173 

173+3*F. 95(6, 346)  173+3*1.785 

to  convert  to  A,  ln(.9700)/-3  =  .01016 


.9700 


Phase  I  Extension 


261  +  48 
264  +  50 


.9841  ,  convert  to  A, 
A  =  In ( . 9841) /-3 


.005351 
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Lower 
Limit  = 


to  convert  to 


Upper 
Limit  = 


to  convert  to 


665  +  48 
672  +  50 


Lower 
Limit  = 


(261+48) *F . 95 (2*261+2*48,2*264+2*50-2*261-2*48) 
264+50-261-48+ (261+48) *F.95 (2*261+2*48, 


2*264+2*50-2*261-2*48) 

309*F. 95(618, 10)  309*2.06 

-  - - =  .9922 

5+309*F. 95(618, 10)  5+309*2.06 

A,  In  ( . 9922) /-3  =  .002608 

261  +  48 

261+48+ (264+50-261-48) *F.95 (2*264+2*50-2*261- 

2*43,  2*261+2*48) 

309  309 

- -  - =  .9745 

•309+5*F. 95(10,  618)  309+5*1.618 

A,  In  ( . 9745) /-3  =  .008615 


Combined  Phases 


-  =  .9875  ,  convert  to  A, 

A  =  ln(.98.'4)/-3  =  .004181 


(665+48) *F. 95 (2*665+2*48,2*672+2*50-2*665-2*48) 


672+50-665-48+ (665+48) *F . 95 (2*665+2*48, 


2*672+2*50-2*665-2*48) 

713*F. 95(1426,  18)  713*1.66 

- -  -  =  .9925 

9+713*F.95 (1426, 18)  9+713*1.66 
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to  convert  to  A,  ln(.9925)/-3  =  .002525 


Upper  665  +  48 

Limit  =  - 

665+48+ (672+50-665-48) *F. 95(2*672+2*50-2*665- 


2*48,  2*665+2*48) 

713  713 

713+9*F.95  (18,  1426)  713+9*1.448 


.9821 


to  convert  to  A,  ln(.9821)/-3 


.006038 


Bayesian  Expert  Information  Exponential  TBPI  Calculations 


Phase  I 


t(p(B|Yl> 

0.00018 

0.018?// 

0.241852 

0.545598 

0.168585 

0.025009 

Phase  II 

Kp(BJrl) 

0.000576 

0.025328 

0.244706 

0.52407 

0.17491 

0.030411 

Phase  I  Extension 

E(  p( B| r 1 ) 

0.000149 

0.016331 

0.227239 

0.547102 

0.180416 

0.028S64 

Combined 

Phases 

S(p(0|Yl> 

3.65E-0S 

0.016529 

0.292403 

0.56561 

0.115761 

0.009661 

Yi 

Ph  1 

x«26 

Ph  1-ext 

x«26 

Ph  2 

X-27 

Combined 

x*23 

0.0005 

0.00018 

0.180253 

0.000149 

0.175747 

0.000576 

0.221668 

3.65E-05 

0.133S26 

0.0006 

0.00018 

9.21242 

0.000149 

0.198456 

0.000576 

0.246081 

3.658-05 

0.162763 

0.000? 

0.00018 

0.236587 

0.000149 

0.221165 

0.000576 

0.270494 

3.6SE-05 

0.192 

0.0008 

0.00018 

0.260755 

0.000149 

0.243874 

0.000576 

0.322844 

3.658-05 

0.221236 

0.0008 

0.00018 

0.315296 

0.000149 

0.29857 

0.000576 

0.375193 

3.65E-05 

0.250473 

0.001 

0.00018 

0.369838 

0.000149 

0.353265 

0.000576 

0.427542 

3.658-05 

0.27971 

0.0011 

0.00018 

0.42438 

0.000149 

0.40796 

0.000576 

0.479892 

3.65E-05 

0.308946 

0.0012 

0.00018 

0.476922 

0.000149 

0.462655 

0.000576 

0.532241 

3 .658-05 

0.365504 

0.0013 

0.00018 

0.533463 

0.000149 

0.  SI 7 351 

0.000576 

O.S84S91 

3.65E-0S 

0.422061 

0.0014 

0.00018 

0.580005 

0.000149 

0.572046 

0.000576 

0.63694 

3.658-05 

0.478619 

0.0015 

0.018777 

0.642547 

0.016531 

0.626741 

0.025328 

0.68929 

0.016529 

0.535176 

0.0016 

0.01877/ 

0.695229 

0.016531 

0.679798 

0.025328 

0.739164 

0.016579 

0.590084 

0.0017 

0.018777 

0.747911 

0.016S31 

0.732855 

0.025328 

0.789038 

0.016529 

0.644992 

0.0018 

0.018777 

0.800593 

0.016531 

0.785913 

0.025328 

0.803996 

0.016529 

0.6999 

0.0019 

0.01877? 

0.815574 

0.016531 

0.802301 

0.025328 

0.818954 

0.016529 

0.754809 

0.002 

0.018777 

0.830555 

0.016531 

0.81869 

0.025328 

0.833912 

0.016529 

0.809717 

0.0021 

0.018777 

0.845535 

0.016531 

0.835078 

0.025328 

0.848871 

0.016529 

0.864625 

0.0022 

0.018777 

0.860516 

0.016531 

0.851467 

0.025328 

0.863829 

0.016529 

0.874548 

0.0023 

0.018777 

0.875497 

0.016531 

0.867855 

0.025328 

0.878787 

0.016529 

0.884471 

0.0024 

0.018777 

0.890478 

0.016531 

0.884244 

0.025328 

0.893745 

0.016529 

0.894394 

0.0025 

0.241852 

0.90S459 

0.227239 

0.900632 

0.244706 

0.908703 

0.292403 

0.904318 

0.0076 

0.241852 

0.898132 

0.227239 

0.89535 

0.244706 

0.901724 

0.292403 

0.886653 

0.0027 

0.241852 

0.890005 

0.227239 

0.891267 

0.244706 

0.894744 

0.292403 

0.868989 

0.0028 

0.241852 

0.883479 

0.227239 

0.886585 

0.244706 

0.873315 

0.292403 

0.851325 

0.0029 

0.241852 

0.861794 

0.227239 

0.866/18 

0.244706 

0.851885 

0.292403 

0.833661 

0.003 

0.241852 

0.84011 

0.227239 

0.84685 

0.244706 

0.830456 

0.292403 

0.815996 

0.0031 

0.241852 

0.818426 

0.227239 

0.826982 

0.244706 

0.809026 

0.292403 

0.798332 

0.0032 

0.241852 

0.796742 

0.227239 

0.8071 t5 

0.244706 

0.787597 

0.292403 

0.770058 

0.0033 

0.241852 

0.775057 

0.227239 

0.787247 

0.244706 

0.766167 

0.292403 

0.741784 

0.0034 

0.241852 

0.733373 

0.227239 

0.76738 

0.244706 

0.744738 

0.292403 

0.713509 

0.0035 

0.543598 

0.731689 

0.547102 

0.747512 

0.52407 

0.723309 

0.56561 

0.685235 

0.0036 

0.S4S598 

0.67963 

0.547102 

0.695658 

0.52407 

0.673943 

0.56561 

0.62964 

0.0037 

0.545598 

0.62/371 

0.547102 

0.643805 

0.52407 

0.624577 

0.56561 

0.574045 

0.0038 

0.545598 

0.573312 

0.547102 

0.591951 

0.52407 

0.57217 

0.56561 

0.51845 

0.0039 

0.S45598 

0.S20952 

0.547102 

0.537241 

0.52407 

0.519763 

0.56561 

0.462855 

0.004 

0.545S98 

0.466393 

0.547102 

0.48253 

0.52407 

0.467356 

0.56561 

0.40726 

0.0041 

0.545598 

0.411833 

0.547102 

0.42782 

0.5240? 

0.414949 

0.56561 

0.351666 

0.0042 

0.545598 

0.357273 

0.547102 

0.37311 

0.52407 

0.362542 

0.56561 

0.295105 

0.0043 

0.545598 

0.302713 

0.547102 

0.3184 

0.52407 

0.31013S 

0.56561 

0.238543 
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0.0044  0.545598  0.2481S4  0.547102  0.26369  0.52407  0.257728  0.56561  0.181982 

0.0045  0.168585  0.193594  0.180416  0.20898  0.17491  0.205321  0.115761  0.125421 

0.0C46  0.168SU5  0.176735  0.180416  0.190938  0.17491  0.18783  0.115761  0.113845 

0.0047  0.168585  0.1S98/7  0.180416  0.172896  0.17491  0.170339  0.115761  0.102269 

0.0048  0.168585  0.143018  0.180416  0.154855  0.17491  0.1S2848  0.115761  0.090693 

0.0049  0.168585  0.12616  0.180416  0.136813  0.17491  0.13S3S7  0.11S761  0.079117 

0.005  0.168585  0.109301  0.180416  0.118772  0.17491  0.117866  0.115761  0.067541 

0.0051  0 . 168585  0.092443  0.180416  0.10073  0.17491  0.100375  0.11S761  0.0S596S 

0.0052  0.168585  0.075584  0.180416  0.082688  0.17491  0.082884  0.115761  0.044389 

0.00S3  0.16858S  0.058726  0.180416  0.064647  0.17491  0.065393  0.US761  0.032813 

0.0054  0.16858.5  0.041867  0.180416  0.046605  0.17491  0.047902  0.115761  0.021237 

0.0055  0 . 025009  0.025009  0.028564  0.028564  0.030411  0.030411  0.009661  0.009661 

0.0056  0.025009  0.022508  0.028564  0.025707  0.030411  0.02737  0.009661  0.008695 

0.00S7  0.02S009  0.020007  0.028564  0.0228S1  0.030411  0.024329  0.009661  0.007729 

0.0058  0.025009  0.017506  0.028564  0.019995  0.030411  0.021288  0.009661  0.006763 

0.0059  0.025009  0.015005  0.028564  0.017138  0.030411  0.018247  0.009661  0.005797 

0.006  0.025009  0.012S04  0.028564  0.014282  0.030411  0.015205  0.009661  0.00483 

0.0061  0.02S009  0.010004  0.028564  0.011425  0.030411  0.C12164  0.009661  0.003864 

0.0062  0.025009  0.007503  0.028564  0.008569  0.030411  C. 009123  0.009661  0.002898 

0.0063  0.025009  0.005002  0.028564  0.005713  0.030411  0.006082  0.009661  0.001932 

0.0064  0.025009  0.002501  0..028S64  0.002856  0.030411  0.003041  0.009661  0.000966 
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Bayesian  Expert  Information  Binomial  TBP1  Calculations 


Phase  I 


t(p(0|Yl) 

2.03E-05 

0.00423 

0.109058 

0.492242 

0.304215 

0.090234 

Phase  II 

X(P<ft|Yl) 

0.000214 

o 

o 

on 

o 

o 

0.174326 

0.516565 

0.238501 

0 .OS  7 355 

Phase  I  Extension 

S(p(B| Yi) 

1.41E-05 

0.004053 

0.106385 

0.488931 

0.307683 

0.092929 

Coeblned 

Phases 

S(p(B| vl) 

2.01E-07 

<0 

O 

O 

o 

• 

o 

0.040409 

0.391317 

0.40066 

0.167158 

Yi 

Ph  1 

x«29 

Ph  1-ext 

X*29 

Ph  2 

X*28 

Coeblned 

x-26 

0.0005 

2.0  IE-05 

0.113308 

1.91E-0S 

0.110457 

0.000214 

0.170146 

2.018-07 

0.028742 

0.0006 

2.03E-05 

0.16253 

1.91E-05 

0.159348 

0.000214 

0.187558 

2.018-07 

0.032783 

0.0007 

2.03E-05 

0.211753 

1.91E-05 

0.208239 

0.000214 

0.239193 

2.018-07 

0.036824 

o.oooe 

2.03E-05 

0.260975 

1.91E-05 

0.257131 

0.000214 

0.290828 

2.018-07 

0.040865 

0.0009 

2.038-05 

0.310197 

1.918-05 

0.306022 

0.000214 

0.342463 

2.018-07 

0.079997 

0.001 

2.03E-05 

0.359419 

1.91E-05 

0.354913 

0.000214 

0.394098 

2.018-07 

0.119128 

0.0011 

2.03E-05 

0.408641 

1 .918-05 

0.403804 

0.000214 

0.445733 

2.018-07 

0.15826 

0.0012 

2.03E-05 

0.457864 

1.918-05 

0.452695 

0.000214 

0.497368 

2. 018-07 

0.197392 

0.0013 

2.03E-05 

0.507086 

1.918-05 

0.501587 

0.000214 

0.549003 

2.018-07 

0.236524 

0.0014 

2.03E-05 

0.556308 

1.918-05 

0.550478 

0.000214 

0.600638 

2.018-07 

0.27S655 

0.0015 

0.00423 

0.60553 

0.004053 

0.599369 

0.013039 

0.652273 

0.000456 

0.314787 

0.0016 

0.00423 

0.635529 

0.0040S3 

0.629732 

0.013039 

0.702626 

0.000456 

0.353873 

0.0017 

0.00423 

0.665527 

0.004053 

0.660095 

0.013039 

0.725172 

0.000456 

0.392959 

0.0018 

0.00423 

0.695526 

0.004053 

0.690458 

0.013039 

0.747718 

0.000456 

0.432045 

0.0019 

0.00423 

0.725524 

0.004053 

0.720821 

0.013039 

0.770265 

0.0004S6 

0.472066 

0.002 

0.00423 

0.755523 

0.004053 

0.751184 

0.013039 

0.792811 

0.000456 

0.512086 

0.0021 

0.00423 

0.785521 

0.004053 

0.781547 

0.013039 

0.815357 

0.000456 

0.552107 

0.0022 

0.00423 

0.81552 

0.004053 

0.81191 

0.013039 

0.837904 

0.000456 

0.592127 

0.0023 

0.00423 

0 . 845518 

0.004053 

0.842273 

0.013039 

0.86045 

0.000456 

0.632147 

0.0024 

0.00423 

0.875517 

0.004053 

0.872636 

0.013039 

0.882996 

0.000456 

0.672168 

0.0025 

0.109058 

0.905515 

0.106385 

0.902999 

0.174326 

0.905542 

0.040409 

0.712188 

0.0026 

0.109058 

0.903633 

0.106385 

0.901653 

0.174326 

0.91196 

0.040409 

0.748213 

0.0027 

0.109058 

0.90175 

0.106385 

0.900307 

0.174326 

0.900263 

0.040409 

0.784238 

0.0028 

0.109058 

0.899868 

0.106385 

0.898962 

0.1743?^ 

0.888566 

0.040409 

0.820264 

0.0029 

0.109058 

0.897986 

0.106385 

0.897616 

0.174326 

0.876868 

0.040409 

0.832938 

0.403 

0.109058 

0.896103 

0.106385 

0.896271 

0.174326 

0.865171 

0.040409 

0.845613 

0.0031 

0.109058 

0.894221 

0.10638S 

0.894925 

0.174326 

0.853474 

0.040409 

0.858288 

0.0032 

0.10905B 

0.892339 

0.106385 

0.89358 

0.174326 

0.841777 

0.040409 

0.870963 

0.0033 

0.109058 

0.890456 

0.106385 

0.892234 

0.174326 

0.83008 

0.040409 

0.883638 

0.0034 

0.109058 

0.888574 

0.106385 

0.890889 

0.174326 

0.818383 

0.040409 

0.896313 

0.0035 

0.492242 

0.886692 

0.488931 

0.889543 

0.S1656S 

0.806686 

0.391317 

0.908988 

0.0036 

0.492242 

0.837468 

0.488931 

0.84065 

0.516565 

0.760765 

0.391317 

0.886572 

0.0037 

0.492242 

0.788243 

0.488931 

0.791757 

0.51656S 

0.709108 

0.391317 

0.864156 

0.0030 

0.492242 

0.739019 

0.438931 

0.742864 

0.516565 

0.657452 

0.391317 

0.84174 

0.0039 

0.492242 

0.689795 

0.488931 

0.693971 

0.516565 

0.605795 

0.391317 

0.802608 

0.004 

0.492242 

0.640571 

0.488931 

0.645078 

0.516565 

0.554139 

0.391317 

0.763476 

0.0041 

0.492242 

0.591346 

0.488931 

0.596184 

0.S1656S 

0.502482 

0.391317 

0.724345 

0.0042 

0.492242 

0.542122 

0.488931 

0.547291 

0.516565 

0.450826 

0.391317 

0.685213 

0.0043 

0.492242 

0.492898 

0.488931 

0.498398 

0.516565 

0.399169 

0.391317 

0.646081 

0.0044 

0.492242 

0.443674 

0.488931 

0.449505 

0.516565 

0.347513 

0.391317 

0.606949 

0.0045 

0.30421S 

0.39445 

0.307683 

0.400612 

0.238501 

0.295856 

0.40066 

0.567818 
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0.0046  0 . 304215  0.364028  0.307683  0.369844  0.238501  0.272006  0.40066  0.527752 

0.0047  0.304215  0.333607  0.30/683  0.339075  0.238S01  0.248156  0.40066  0.487686 

0.0048  0.304215  0.30318S  0.307683  0.308307  0.238501  0.224306  0.40066  0.44762 

0.0049  0.304215  0.272763  0.307683  0.277539  0.238501  0.200456  0.40066  0.407554 

0.005  0.304215  0.242342  0.307683  0.246771  0.238501  0.176605  0.40066  0.367488 

0.0051  0.304215  0.21192  0.307683  0.216002  0.238501  0.1527SS  0.40066  0.327422 

0.0052  0.304215  0.181499  0.307683  0.185234  0.238501  0.128905  0.40066  0.287356 

0.0053  0.304215  0.151077  0.307683  0.154466  0.238501  0.10S055  0.40066  0.24729 

0.0054  0 . 304215  0.120656  0.307683  0.123698  0.238501  0.08120S  0.40066  0.207224 

0.0055  0.090234  0.090234  0.092929  0.092929  0.057355  0.0573S5  0.167158  0.167158 

0.0056  0.090234  0.081211  0.092929  0.083636  0.057355  0.051619  0.167158  0.150442 

0.005/  0.090234  0.072187  0.092929  0.074344  0.0S73S5  0.045884  0.167158  0.133726 

0.0058  0.090234  0.063164  0.092929  0.065051  0.057355  0.040148  0.167158  0.11701 

0.00S9  0.090234  0.054141  0.092929  0.0SS758  0.0S7355  0.034413  0.167158  0.10029S 

0.006  0.090234  0.045117  0.092929  0.046465  0.057355  0.028677  0.167158  0.083579 

0.0061  0.090734  0.036094  0.092929  0.037172  0.057355  0.022942  0.167158  0.066863 

0.0062  0.090234  0.02707  0.092929  0.027879  0.057355  0.017206  0.167158  0.050147 

0.0063  0.090234  0.018047  0.092929  0.018586  0.057355  0.011471  0.167158  0.033432 

0.0064  0.090234  0.009023  0.092929  0.009293  0.057355  0.005735  0.167158  0.016716 
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